A Glycemia Risk Index (GRI) of Hypoglycemia and Hyperglycemia for Continuous Glucose Monitoring Validated by Clinician Ratings

Abstract

Background:

A composite metric for the quality of glycemia from continuous glucose monitor (CGM) tracings could be useful for assisting with basic clinical interpretation of CGM data.

Methods:

We assembled a data set of 14-day CGM tracings from 225 insulin-treated adults with diabetes. Using a balanced incomplete block design, 330 clinicians who were highly experienced with CGM analysis and interpretation ranked the CGM tracings from best to worst quality of glycemia. We used principal component analysis and multiple regressions to develop a model to predict the clinician ranking based on seven standard metrics in an Ambulatory Glucose Profile: very low–glucose and low-glucose hypoglycemia; very high–glucose and high-glucose hyperglycemia; time in range; mean glucose; and coefficient of variation.

Results:

The analysis showed that clinician rankings depend on two components, one related to hypoglycemia that gives more weight to very low-glucose than to low-glucose and the other related to hyperglycemia that likewise gives greater weight to very high-glucose than to high-glucose. These two components should be calculated and displayed separately, but they can also be combined into a single Glycemia Risk Index (GRI) that corresponds closely to the clinician rankings of the overall quality of glycemia (r = 0.95). The GRI can be displayed graphically on a GRI Grid with the hypoglycemia component on the horizontal axis and the hyperglycemia component on the vertical axis. Diagonal lines divide the graph into five zones (quintiles) corresponding to the best (0th to 20th percentile) to worst (81st to 100th percentile) overall quality of glycemia. The GRI Grid enables users to track sequential changes within an individual over time and compare groups of individuals.

Conclusion:

The GRI is a single-number summary of the quality of glycemia. Its hypoglycemia and hyperglycemia components provide actionable scores and a graphical display (the GRI Grid) that can be used by clinicians and researchers to determine the glycemic effects of prescribed and investigational treatments.

Introduction

Continuous glucose monitor (CGM) data is emerging as a useful tool for assessing and quantifying the quality of glycemic control. ¹ Glycemic control encompasses the risk of both acute hypoglycemia and chronic hyperglycemia, ² which is in turn associated with long-term complications. ³ Clinicians and patients would benefit from CGM metrics that characterize the proportions of time with both very low/low, and high/very high glucose concentrations, a concept which can be termed “quality of glycemia.”

A widely used report recommended in the American Diabetes Association (ADA) Standards of Medical Care in Diabetes–2022 ⁴ for summarizing the results of CGM tracings is the Ambulatory Glucose Profile (AGP), ⁵ which presents seven key metrics from a CGM tracing.^6,7 These metrics include the following:

Percentages of time in:

1. very low–glucose hypoglycemia (VLow: <54 mg/dL; <3.0 mmol/L) (level 2 hypoglycemia)

2. low-glucose hypoglycemia (Low: 54-<70 mg/dL; 3.0-<3.9 mmol/L) (level 1 hypoglycemia)

3. target range (TIR: 70-180 mg/dL; 3.9-10.0 mmol/L),

4. high-glucose hyperglycemia (High: >180-250 mg/dL; >10.0-13.9 mmol/L) (level 1 hyperglycemia)

5. very high–glucose hyperglycemia (VHigh: >250 mg/dL; >13.9 mmol/L) (level 2 hyperglycemia) as well as

6. coefficient of variation (CV) (standard deviation /mean glucose)

7. mean glucose (MG) — as well as the glucose management indicator (GMI), which is a measure linearly related to MG ⁵

These seven metrics are highly interdependent. For example, the five percent-of-time metrics cover 100% of the monitoring period so that any one of them can be determined by subtracting the other four from 100%. The time in very low-glucose (VLow) plus the time in low-glucose (Low) is the time below range (TBR). The time in very high-glucose (VHigh) plus the time in high-glucose (High) is the time above range (TAR). In an AGP report, these seven metrics are each typically presented with a target range. To interpret a CGM profile, a clinician must simultaneously process these seven metrics, along with an aggregated 14-day glucose profile (but other measurement durations could also be used now or in the future) to determine the quality of glycemia. Interdependence means that if a clinician tries to improve one metric, for example, TIR, then other metrics might improve or worsen. This makes the treatment optimization task difficult and unpredictable because it is unlikely for all metrics associated with an AGP to improve simultaneously.

Metrics for Glycemic Control

The ADA Standards of Care state that the TIR as measured by a CGM can be used for assessment of glycemic control. ⁴ Many clinicians use this single number as a guide to the quality of a patient’s glycemia. However, use of TIR in this context has been criticized for not being adequately sensitive to hypoglycemia.^8,9 As an alternative to TIR, several composite scores have been proposed to combine measures of glycemic control.^9,10 However, some of these scores may not adequately reflect both hypoglycemia and hyperglycemia or provide greater weighting for time in VLow than time in Low, or for time in VHigh than time in High. ¹¹

In this study, we (1) identify the two essential components that best present a person’s glycemic state—one responsible for the risk of hypoglycemia and the other responsible for exposure to hyperglycemia, based on a graphical and numerical interpretation of AGP data, and (2) introduce a composite metric that describes the quality of a CGM wearer’s glycemic control in a single score weighted according to the risk for hypoglycemia and hyperglycemia, based on their importance as systematically evaluated by a large number of experienced clinicians. Such a metric would provide clinicians with a single number accounting for the principal dimensions of their patients’ glycemic control and would facilitate review of multiple CGM reports over time.

Methods

Design

We developed a model to predict the rankings by experienced clinicians of 14-day CGM tracings. The study was approved by the University of Texas San Antonio Health Science Center Institutional Review Board.

Data Set of Continuous Glucose Monitor Tracings

We assembled a de-identified data set of 14-day CGM tracings from 225 adults with diabetes that had been used in the DIAMOND, ¹² DCLP3, ¹³ REPLACE-BG, ¹⁴ and DIAMOND T2D ¹⁵ Trials. The CGMs used were Dexcom G4 Platinum CGM Systems with an enhanced algorithm (software 505)^12,14,15 and Dexcom G6 sensors ¹³ (Dexcom Inc., San Diego, CA, USA) with the same format for displaying trend data. The CGM tracings were from subjects in four categories: type 1 diabetes (T1D) using (1) hybrid closed loop (HCL), (2) insulin infusion pump (Pump), (3) multiple daily insulin injections (MDI), or (4) type 2 diabetes (T2D) using MDI. The types of subjects in each of the four trials included the following: DIAMOND ¹² (T1D MDI users—blinded run-in for 14 days); DCLP3 ¹³ (T1D MDI users—either prestudy personal CGM data for MDI users using Dexcom CGM or unblinded run-in data for 14 days collected while using MDI prior to starting Pump, and T1D HCL users—unblinded CGM while using Control IQ [Tandem, San Diego, CA, USA]); REPLACE-BG ¹⁴ (T1D Pump users—baseline data which was either prestudy personal CGM or run-in phase blinded or unblinded CGM), and DIAMOND T2D ¹⁵ (T2D MDI users—blinded baseline data). Each patient category was represented by 56 CGM tracings, except for T2D, which had 57 CGM tracings. Every CGM tracing contained at least 92% of the potential data points.

Clinician Rankers

We invited 330 expert diabetologists from six continents who reported reviewing at least 20 CGM tracings per month in their clinical practice. These clinicians ranked the 225 CGM tracings from best to worst in terms of the quality of their glycemia. The geographic locations and specialties of the clinicians are presented in Supplementary Tables S1a and S1b.

Ranking Process

In pretesting, we determined that each clinician could compare five CGM tracings at a time, so we used an incomplete block design with a block size of five.^16-18 For the incomplete block design to be balanced, each possible pair of CGM tracings should appear in the same number of blocks, such that any given 14-day CGM tracing is compared against all other CGM tracings the same number of times—in this case, two times. Details of the balanced, incomplete block design and assessment of inter-rater agreement are provided in the Supplementary Material. Ultimately, each of the 225 CGM tracings appeared in 22 separate blocks, and each received 22 independent rankings on a scale ranging between 0 and 4. The rankings were based on the CGM tracings alone without additional instructions. The format for the CGM tracings was identical, regardless of CGM make/model. The 225 CGM tracings were sorted by average ranking, normalized so that each tracing was assigned a percentile score ranging from 0 to 100, with 0 representing “best” or “no risk” and 100 representing “worst” or “maximum risk.” This clinicians’ percentile ranking was the quantity that the new composite index, the Glycemia Risk Index (GRI), was intended to predict. We then sought to find the best model to predict the clinicians’ percentile rankings for each subject using the seven available CGM metrics from an AGP report.

Modeling Approach

To uncover the essential components of glycemic control, we applied principal component analysis (PCA) for dimensionality reduction and variable selection. We also compared each individual metric to the clinician rankings. To develop the model, we used five-fold cross-validation with recursive feature elimination to determine the optimal number of metrics to include and to inform variable selection. ¹⁹ In addition, we used linear regression–based variable selection approaches, including forward, backward, stepwise, and Lasso regressions. ²⁰ We also used two machine learning feature selection methods, Boruta ²¹ and recursive partitioning,^22,23 to evaluate the relative importance of input variables. We considered nonlinear relationships, while endeavoring to achieve a balance between model simplicity and goodness-of-fit.

Statistical Analysis

We summarize each of the seven AGP metrics with mean, standard deviation (SD), median, 25th/75th percentile, minimum, and maximum, calculated for the entire set of 225 CGM tracings and for the CGM tracings stratified by the four patient categories. For all bivariate comparisons between metrics, we display scatterplots and report correlation coefficients. The PCA results are summarized in a correlation matrix. We then consider the clinicians’ percentile rankings, reporting bivariate comparisons between each metric and clinician rankings using scatterplots and correlation coefficients. Finally, we present the newly proposed metric, GRI. Goodness-of-fit for alternative models is summarized with the correlation coefficient, adjusted R² (Adj R²), and root mean square error (RMS). We evaluated goodness-of-fit for all subjects combined and for each of the four patient categories. We evaluated statistical significance of models involving different numbers of input variables (regression coefficients) and degrees of freedom using the extra sum of squares principle. ²⁴

Results

Data Set of Continuous Glucose Monitor Tracings

The 225 CGM tracings showed wide variation in all seven AGP metrics. For example, MG ranged from 94 to 267 mg/dL (5.2-14.8 mmol/L) and VHigh ranged from 0% to 57% of the time (Table 1, Supplementary Figure S1). The category of persons with T1D using MDI had the worst quality of glycemia as evidenced by highest average time in VLow (2.5%), lowest average TIR (43.9%), highest average time in VHigh (23.7%), and highest MG (188 mg/dL; 10.4 mmol/L) (Supplementary Table S2).

Table 1.

Summary of Ambulatory Glucose Profile Metrics for 225 Continuous Glucose Monitor Tracings.

	Overall (N = 225)		Min.	Percentile			Max.
	Avg	SD	0th	25th	50th	75th	100th
% of time
Very Low (<54 mg/dL; <3.0 mmol/L)	1.2	2.0	0	0	0	1	12
Low (54-<70 mg/d; 3.0-<3.9 mmol/L)	2.5	2.8	0	0	2	4	15
In Range (70-180 mg/dL; 3.9-10.0 mmol/L)	59.9	21.2	13	42	59	80	97
High (>180-250 mg/dL; > 10.0-13.9 mmol/L)	23.4	11.5	1	14	23	32	58
Very High (>250 mg/dL; >13.9 mmol/L)	13.0	12.9	0	2	10	21	57
Total	100.0
Other
Mean Glucose (mg/dL)	167	34	94	140	161	192	267
Coefficient of Variation	0.35	0.08	0.18	0.29	0.33	0.40	0.62

Principal Component Analysis

The PCA showed that the seven metrics can be divided into two distinct, highly correlated groups or clusters: a hypoglycemia-related group (including VLow, Low, and CV) and a hyperglycemia-related group (including VHigh, High, TIR, and MG) (Figure 1). These two clusters, which involve three and four metrics (hypoglycemia and hyperglycemia, respectively), explain 88% of the variance in the clinicians’ percentile rankings for all 225 CGM tracings. Thus, two essential components account for nearly 90% of the variability in the seven AGP metrics.

Figure 1.

Correlation between pairs of metrics, where 1.0 indicates a strong correlation and −1.0 indicates a strong inverse correlation. The principal component analysis showed that the seven metrics divide into two highly correlated groups: a hypoglycemia-related group (including VLow, Low, and CV) and a hyperglycemia-related group (including VHigh, High, TIR, and MG). Therefore, each group of metrics is represented by one principal dimension, or essential component, of the quality of glycemia (the hypoglycemia component and the hyperglycemia component). Abbreviations: VLow, <54 mg/dL; <3.0 mmol/L; Low, 54-<70 mg/dL; 3.0-<3.9 mmol/L; CV, coefficient of variation (standard deviation of glucose/mean glucose); VHigh, >250 mg/dL; >13.9 mmol/L; High, >180-250 mg/dL; >10.0-13.9 mmol/L; TIR, time in target range (70-180 mg/dL; 3.9-10.0 mmol/L); MG, mean glucose (mg/dL).

A criticism of TIR as a single metric is that it is not sensitive to hypoglycemia. ⁸ In our CGM data set, the correlation between TIR and VLow was −0.11 (Figure 1) and the correlation between TIR and TBR (VLow + Low) was found to be 0.01.

Development of the GRI

The clinicians’ rankings were used to assign each CGM tracing a percentile score ranging from 0 to 100, with 0 representing “best” or “no risk” and 100 representing “worst” or “maximum risk.” This clinicians’ percentile ranking was the quantity that we sought to predict with the GRI.

Multifold cross-validation determined that the optimal number of features, balancing bias and variance, was four. Because the AGP’s five percent-of-time metrics sum to 100%, they are exactly collinear. A linear regression model that includes two or more exactly collinear variables will not produce a solution. Therefore, it was necessary to remove one of the five percent-of-time metrics from our model so that the four remaining variables would not add up to 100%. Ultimately, we removed TIR. This is because TIR is highly negatively correlated (r = −0.91) with the VHigh metric, so only one of the two should be included in the model. Both forward and backward stepwise regression retained VHigh in preference to TIR. We also used two machine learning feature selection methods, Boruta and recursive partitioning; both methods indicated that VHigh was more important than TIR. Although not specifically included in our model, TIR is included implicitly since it can be calculated from the other four percent-of-time metrics: TIR = 100% – (VHigh + High + Low + VLow). MG was highly correlated with the other hyperglycemia metrics, but was the least important metric in several variable selection methods including recursive feature elimination, forward and backward stepwise regression, and LASSO regression. Similarly, the CV was highly correlated with the hypoglycemia metrics. Of the three correlated hypoglycemia metrics, we chose to retain VLow and Low in preference to the CV because VLow and Low are more clearly actionable metrics. The CV is the ratio of the SD of the glucose concentrations (a marker of variability with a known relationship to hypoglycemia) divided by MG (an indirect marker of hyperglycemia). Whereas both SD and MG will be lower with optimal treatment, their ratio would not necessarily decrease. Furthermore, the variance inflation factor (VIF) associated with retaining the CV in the model was 13.7. This VIF greater than 10 indicates that CV was strongly correlated with the other four metrics (Low, VLow, High, and VHigh) and could be eliminated from the model. ²¹ After eliminating TIR, MG, and CV, we were left with Low, VLow, High, and VHigh as the four essential parameters (independent variables) in our model. These four metrics are well established and clinically actionable. ⁵

Using both the PCA results and clinical reasoning, we combined VLow and Low into a unified hypoglycemia component, and similarly, VHigh and High into a unified hyperglycemia component. For both of these essential components, we assigned full weight (coefficient of 1) to the percent of time with the more extreme abnormality and a reduced weight (coefficient < 1) to the percent of time with the less extreme abnormality. This is a common weighting technique that has been used with glucose data for more than 20 years. ²⁵ By construction, neither component can exceed 100%. Based on the ratio of the coefficients in the linear regression model with the clinicians’ percentile ranking as the dependent variable and VLow, Low, High, and VHigh as the independent variables, we determined a weight for Low of 0.8 of VLow (Table 2, Equation #1), and a weight for High of 0.5 of VHigh (Table 2, Equation #2). Since these coefficient values were based on the ratio of the best-fit regression coefficients, they reflect the combined judgment of the clinicians about the relative importance of less extreme versus more extreme abnormalities. The GRI is then calculated as a linear combination of the two essential components, which represent hypoglycemia and hyperglycemia. Using a linear regression model with the clinicians’ percentile ranking as the dependent variable and the hypoglycemia and hyperglycemia components as the independent variables, we found the best-fit coefficients for the hypoglycemia and hyperglycemia components to be 3.0 and 1.6, respectively. The model formula based on predicted clinician rankings is presented in Table 2.

Table 2.

Formula for Calculating the Glycemia Risk Index.

VLow = very low–glucose hypoglycemia (% of time)
Low = low-glucose hypoglycemia (% of time)
VHigh = very high–glucose hyperglycemia (% of time)
High = high-glucose hyperglycemia (% of time)
Hypoglycemia Component = VLow + (0.8 × Low)	(Equation #1)
Hyperglycemia Component = VHigh + (0.5 × High)	(Equation #2)
GRI = (3.0 × HypoComponent) + (1.6 × HyperComponent)	(Equation #3)
Equivalently,
GRI = (3.0 × VLow) + (2.4 × Low) + (1.6 × VHigh) + (0.8 × High)	(Equation #4)
Example: VLow = 5%, Low = 10%, VHigh = 15%, High = 20%
HypoComponent = 5% + (0.8 × 10%) = 13%	(Equation #1)
HyperComponent = 15% + (0.5 × 20%) = 25%	(Equation #2)
GRI = (3.0 × 13%) + (1.6 × 25%) = 79	(Equation #3)
Equivalently,
GRI = (3.0 × 5%) + (2.4 × 10%) + (1.6 × 15%) + (0.8 × 20%) = GRI = 15 + 24  + 24  + 16  = 79
In this example
TIR = 100% – (VLow + Low + VHigh + High)
= 100% – (5% + 10% + 15% + 20%)
= 100 – 50%
= 50%

Abbreviations: GRI, Glycemia Risk Index; VLow, very low–glucose hypoglycemia (<54 mg/dL; <3.0 mmol/L) (level 2 hypoglycemia); Low, low-glucose hypoglycemia (54-<70 mg/dL; 3.0-<3.9 mmol/L) (level 1 hypoglycemia); TIR, time in target range (70-180 mg/dL; 3.9-10.0 mmol/L); High, high-glucose hyperglycemia (>180-250 mg/dL; >10.0-13.9 mmol/L) (level 1 hyperglycemia); VHigh, very high–glucose hyperglycemia (>250 mg/dL; >13.9 mmol/L) (level 2 hyperglycemia).

This model was designed to give a best-fit, unbiased estimate of the clinicians’ percentile rankings, meaning that it fits the rankings well and is equally likely to underestimate or overestimate the ranking (Figure 2). Although the GRI’s hypoglycemia and hyperglycemia components cannot exceed 100%, a calculated GRI exceeding 100 is arithmetically possible if a CGM tracing would receive a high clinician ranking close to 100 with overestimation by the model. If Equation #4 exceeds 100, then the GRI is capped at 100.

Figure 2.

Relationship between clinicians’ percentile rankings and the Glycemia Risk Index. Each individual clinician’s data point is represented as a blue dot. The 95% confidence interval is in solid yellow lines and the 95% prediction interval is in the dashed line. Abbreviations: GRI, Glycemia Risk Index; CGM, continuous glucose monitor; RMS, root mean square error.

The GRI predicts the clinicians’ percentile rankings. It is calculated as GRI = (3.0 × VLow) + (2.4 × Low) + (1.6 × VHigh) + (0.8 × High), with a maximum permissible value of 100.

Correlation of the GRI With Other Metrics

Since the GRI was designed to estimate the clinicians’ percentile rankings (with 0 being the best and 100 being the worst), it is not surprising that the GRI fits those rankings better than any of the seven individual metrics reported with the AGP. The correlation between the seven AGP metrics and clinician rankings are illustrated in Figure 3 and Supplementary Figure S2. For example, as TIR decreases, the clinicians’ percentile ranking increases, and compared with the GRI, TIR does not correlate as well with the clinicians’ percentile rankings (TIR: R² = 0.824, RMS = 12.16; GRI: R² = 0.904, RMS = 8.95; P < .00001 by extra sum of squares). When TIR was approximately 50%, the clinicians’ percentile ranking could be as high as 97 or as low as 44. Similarly, when TIR was approximately 70%, the clinicians’ percentile ranking could be as high as 62 or as low as 25 (Figure 3). This discrepancy in the clinicians’ percentile rankings for a given TIR reflects (1) differences in how much of the time out of range was hypoglycemia versus hyperglycemia and (2) the relative contributions of Low, VLow, High, and VHigh. These factors are incorporated into the GRI, but are not reflected in TIR.

Figure 3.

Clinicians’ percentile rankings (lower is better) versus time in range (higher is better). When time in range was approximately 50%, the clinicians’ percentile ranking could be as high as 97 (A) or as low as 44 (B). Similarly, when time in range was approximately 70%, the clinicians’ percentile ranking could be as high as 62 (C) or as low as 25 (D). Abbreviations: RMS, root mean square error; CGM, continuous glucose monitor; TIR, time in target range.

Since the GRI is based on weighted combinations of TBR (the hypoglycemia component) and TAR (the hyperglycemia component), like the clinicians’ percentile rankings, the GRI is high when TIR is low. The correlation coefficient between the GRI and TIR is −0.91 (Figure 1).

TBR (VLow + Low) combined with TIR is an established combination of metrics that can be an effective basis for interventions to improve the quality of glycemia. ²⁶ We considered a model based on an optimized combination of TBR and TIR. However, this model of a composite metric did not fit the clinicians’ rankings quite as well as the GRI in terms of adjusted R² and RMS. (TBR/TIR: Adj R² = 0.889; RMS = 9.64; GRI: Adj R² = 0.904, RMS = 8.95, P < .0001 by extra sum of squares.) ²⁴ The GRI has the added advantage that both components are higher when glycemic control is worse. The ideal value is zero for GRI and both the hypoglycemia and hyperglycemia components. This makes for an intuitive graphical display on a two-dimensional grid as in Figures 4 to 6.

Figure 4.

A Glycemia Risk Index grid showing the hyperglycemia component versus the hypoglycemia component for all 225 CGM tracings. The results for each of the four categories of patients are shown with different symbols. We highlighted individual data points for the CGM tracings from two persons (designated P1 and P2) with type 1 diabetes receiving multiple daily insulin injections. Abbreviations: GRI, Glycemia Risk Index; T1D, type 1 diabetes; MDI, multiple daily insulin injections; VLow, very low–glucose hypoglycemia (<54 mg/dL; <3.0 mmol/L) (level 2 hypoglycemia); Low, low-glucose hypoglycemia (54-<70 mg/dL; 3.0-<3.9 mmol/L) (level 1 hypoglycemia); High, high-glucose hyperglycemia (>180-250 mg/dL; >10.0-13.9 mmol/L) (level 1 hyperglycemia); VHigh, very high–glucose hyperglycemia (>250 mg/dL; >13.9 mmol/L) (level 2 hyperglycemia); TIR, time in target range (70-180 mg/dL; 3.9-10.0 mmol/L); Hypo, Hypoglycemia Component; Hyper, Hyperglycemia Component; Pump, insulin infusion pump; HCL, hybrid closed loop; T2D, type 2 diabetes.

Figure 5.

The Glycemia Risk Index over time for five different time periods. Legend: Between times 1 and 2, the TIR worsened by decreasing from 46% to 40%. However, the GRI improved from 90 to 75. For time 1, the hypoglycemia/hyperglycemia components are 16%/26%. For time 2, they are 6%/35%. Adjustment to reduce hypoglycemia could increase hyperglycemia. Abbreviations: GRI, Glycemia Risk Index; TIR, time in range; Hypo, hypoglycemia component; Hyper, hyperglycemia component.

Figure 6.

The median and IQR for the Hyperglycemia Component and Hypoglycemia Component for the four patient categories and overall. Abbreviations: IQR, interquartile range; GRI, Glycemia Risk Index; T1D, type 1 diabetes; MDI, multiple daily insulin injections; HCL, hybrid closed loop; Pump, insulin infusion pump; T2D, type 2 diabetes.

We separately assessed goodness-of-fit between the GRI and the clinicians’ percentile rankings for CGM tracings from each of the four patient categories. The fit for the individual patient categories was similar to the fit for the overall group.

We evaluated several models for matching the clinicians’ percentile rankings. Overall, the GRI was the best model identified.

Graphical Display of Hyperglycemia Versus Hypoglycemia

Since glycemic control is a two-dimensional quantity, the GRI’s hypoglycemia and hyperglycemia components can be displayed on a two-dimensional plot called the GRI Grid (Figure 4). We chose to display the hypoglycemia component (0%-100%) on the horizontal axis and the hyperglycemia component (0%-100%) on the vertical axis. A set of diagonal lines divides the graph into five glycemia risk zones (which we label A-E) corresponding to the best (first-20th percentile) to worst (81st-100th percentile) quintiles for overall quality of glycemia. We highlighted two points corresponding to CGM tracings from two persons with T1D, whom we will call P1 and P2, both treated with MDI. Both have similar GRIs (and clinicians’ percentile ranking), but one person (P1) had no hypoglycemia on the CGM tracing and the other (P2) had 8% VLow and 7% Low. The TIR for these two CGM tracings was different for P1 versus P2 (31% vs 55%), but because the person with the higher TIR had substantial hypoglycemia, the GRIs are almost the same (76 and 75, respectively).

The grid also enables the user to track sequential changes within an individual. Figure 5 shows a hypothetical patient’s progression over five time periods (time 1-time 5). Between time 1 and time 2, TIR worsened by decreasing from 46% to 40%. However, the GRI between the same two periods improved from 90 to 75. At time 1, the hypoglycemia and hyperglycemia components are 16% and 26%. For time 2, they are 6% and 35%. In this example, the treatment was adapted to reduce the risk of hypoglycemia, resulting in much less hypoglycemia, slightly less time in target range, and more hyperglycemia: VLow decreased from 8% to 3% and Low decreased from 10% to 4%. The amount of time in VLow and Low ranges combined and the amount of time in the VHigh and High ranges were more favorable at time 2 than at time 1, resulting in an improved GRI, even though the TIR worsened. Because GRI accounts for more extreme out-of-range times and assigns greater weight to hypoglycemia than to hyperglycemia, an improvement (reduction) in VLow can have more influence than a worsening in TIR. The divergence (improving GRI and worsening TIR) between time 1 and time 2 in this example was followed by progressive improvement in both the GRI and the TIR for time 3, time 4, and time 5.

Data from a multiday or multiweek CGM tracing can be broken down into separate daily or weekly GRIs. These GRIs can then be plotted on a grid to determine whether patterns of glycemia during specific days or weeks of a measurement period differ from patterns during other time periods.

The mean GRI was lowest in the T1D HCL category (mean GRI = 26), highest in the T1D MDI category (mean GRI = 78) and similar in the T1D Pump (mean GRI = 43) and T2D MDI (mean GRI =52) categories (Figure 6). The T1D Pump and T2D MDI groups had similar GRIs, but the T1D Pump group had more hypoglycemia and less hyperglycemia than the T2D MDI group. For the T1D Pump group, mean hypoglycemia and hyperglycemia components were 3% and 21%. In the T2D MDI group, they were 2% and 28%.

The GRI Grid displays both hypoglycemia risk and hyperglycemia exposure. This plot allows for an individual patient’s risks to be monitored sequentially over time and for a population of patients to be monitored for identifying those who require additional treatment.

Discussion

The GRI is a composite CGM metric of glycemic risk. This index (1) reflects both the essential hypoglycemia and hyperglycemia components, (2) weights extreme hypoglycemia or hyperglycemia more than less extreme hypoglycemia or hyperglycemia, and (3) correlates with clinician rankings more closely than other models we considered, such as TIR or TIR combined with TBR.

A useful feature of the GRI’s hypoglycemia and hyperglycemia components is that they can be plotted together on a grid with the origin for both located in the lower left corner and extreme values in the upper right corner. The GRI can be reported and plotted longitudinally for one patient or cross-sectionally for a group of patients. Using only a single index may be more attractive to some clinicians, but understanding the two actionable dimensions of hypoglycemia and hyperglycemia, when paired with a glucose profile, should permit and facilitate more appropriate adjustments in therapy. Perhaps a GRI above a chosen level should prompt detailed review of the AGP report or referral to a diabetes specialist.

Comparison With Time in Range

TIR is an easy-to-understand, well-established metric for the quality of glycemia. ²⁷ Both TIR and the GRI can be calculated for any desired time period. An obvious difference between the GRI and TIR is that the GRI is higher when glycemia is worse and the TIR is higher when glycemia is better. Both are on a 0 to 100 scale. If the pattern of glycemia is poor, then the grid portrayal of the GRI indicates whether the problem is too much hypoglycemia, too much hyperglycemia, or too much of both. A glucose profile can then be used to determine an action. TIR as a single measurement does not indicate whether the out of range readings are generally too low or too high, and if they are too low or too high, then TIR does not weight (as experienced clinicians do) hypoglycemia as more significant than hyperglycemia. Also, TIR does not weight extreme deviations from the target range more heavily than less extreme deviations.

TIR and TBR can be used together effectively to express the quality of glycemia.^9,26 However, compared with the GRI’s components, use of a combination of TIR and TBR did not provide as good a fit to clinician rankings. The clinicians distinguished between time spent in the very high versus the high glucose range and, to a lesser extent, between time spent in the very low versus the low glucose range. Also, from an intuitive or visual standpoint, we believe that many clinicians would prefer to use a single-index, linear combination of hypoglycemia and hyperglycemia components, both of which are worse when high, than a combination of TBR and TIR, one of which (TBR) is worse when high and one of which (TIR) is worse when low. Moreover, from a control engineering perspective, TIR and TBR do not combine in a well-defined cost function for the purposes of automated optimization to a target set point.

A clinician or researcher might wish to use GRI in addition to TIR as a summary statistic to understand a set of CGM data from a different perspective than that of TIR. A correlation between TIR and long-term complications has been demonstrated. ²⁸ GRI is a new metric and has not yet been studied for its association with outcomes. In view of the high correlation between TIR and GRI for the present data set, it is highly likely that similar high correlations will be found between GRI and long-term complications.

Comparison With Other Metrics of Glycemic Control

Plotting the GRI’s hyperglycemia component (0%-100%) on the vertical axis and hypoglycemia component (0%-100%) on the horizontal axis with a set of diagonal lines creates a graph similar to one created by Rodbard. ²⁹ The diagonal lines on the display of a GRI grid as in Figure 4 predict the clinicians’ percentile rankings, whereas the diagonal lines in Rodbard correspond to specified times in range. Rodbard displayed percent of time <80 mg/dL on the horizontal axis, while the GRI’s hypoglycemic component gives a weight of 0.8 to less extreme hypoglycemia (54-<70 mg/dL) relative to extreme hypoglycemia (<54 mg/dL). Similarly, Rodbard displayed percent of time >180 mg/dL on the vertical axis, while the GRI’s hyperglycemia component uses a lower weight for less extreme hyperglycemia (>180-250 mg/dL) as 0.5 of the weight for extreme hyperglycemia (>250 mg/dL).

The GRI uses a simple five-step weighting function for glucose values using the AGP glucose boundaries of 54, 70, 180, and 250 mg/dL. It weights VLow:Low:VHigh:High in the ratio 3.75:3:2:1. This increased weighting of extreme glucose values is characteristic of risk indexes used by engineers in algorithm development, which typically assign higher penalties for greater deviations from a safe state. In 1997, Kovatchev et al. introduced the Low Blood Glucose Index/High Blood Glucose Index (LBGI/HBGI) based on a smooth risk function to which the GRI step function roughly corresponds (Figure 7).^30,31 A similar kind of risk function was developed by several other investigators.^8,32

Figure 7.

The Glycemia Risk Index weights glucose abnormalities according to a five-step function. The five-step weighting process for the GRI is in contrast to the smooth LBGI/HBGI risk function introduced by Kovatchev ³⁰ and Kovatchev et al., ³¹ Abbreviations: GRI, Glycemia Risk Index; LBGI, low blood glucose index; HBGI, high blood glucose index.

Limitations

As part of this research effort, we invited 80 experts in CGM clinical research and clinical practice from six continents to complete a survey linking quantitative measures with clinical assessments. We asked the experts to create ten zones of clinical performance from worst to best for each of the seven CGM metrics. The zones did not necessarily have to be equal in width. The experts’ ten zones for each metric were then averaged to assign levels of appeal for each of the analytic measures and make it possible to compare one measure with another in terms of quality of glycemia. The subsequent analysis was performed using both the "raw" CGM metrics and the experts’ scaled scores. Since similar regression results were obtained whether we used the “raw” CGM parameters in standard units or the expert-scaled metrics, we decided to use the original measurements. An analysis of the scaling by experts will be published elsewhere.

The data set of CGM tracings used to develop the GRI came from clinical trials that included four different types of insulin-treated adult patients using CGM and may not be representative of tracings in other clinical populations. For example, the GRI would not be applicable to the quality of glycemia in pregnant women or children until it has been validated in these populations. The GRI is not a substitute for looking at individual metrics but rather is a summary or screening score. It can be used either to supplement individual metrics or determine who warrants either review of individual metrics or referral to an expert in optimizing glycemic control. As with each of the summary measures in the AGP, the GRI does not distinguish by time of day. In most cases, specific treatment decisions will require differentiating between daytime and nighttime or between preprandial and postprandial patterns of glycemia, which can be seen in a composite glucose profile or series of daily glucose profiles that make up the composite profile. The GRI is based on the average ratings of experienced clinicians who were specifically recruited for this study by experts in research and clinical use of CGMs from six continents. A different sample of experienced clinicians (whether by geography or specialty) might have produced a different set of rankings. Finally and most importantly, the GRI is based on clinician rankings, not clinical outcomes. Subsequent studies will be needed to determine how well the GRI predicts outcomes.

Applications

The GRI is a statistic expressing the quality of glycemia that is expected to have value in four contexts. These are (1) managing the health care of CGM users in conjunction with an AGP report, (2) identifying individuals within a population who are most in need of further glycemic optimization, (3) developing algorithms for automated insulin dosing systems to quantify glycemic patterns, so clinicians, statisticians, and other researchers can choose which trade-offs to make, and (4) predicting outcomes in long term studies of interventions intended to decrease risks of complications, both according to the GRI score and the hypoglycemia and hyperglycemia components as potentially independent variables.

Conclusion

In conclusion, we have identified a pair of essential components of glycemic control, one related to hypoglycemia and one related to hyperglycemia. We also introduce a composite metric, the GRI, that describes the quality of glycemia in a CGM tracing. The GRI is a single number weighted according to the risk for hypoglycemia and hyperglycemia and based on the opinions of experienced clinicians. GRI has the potential to become established as a useful statistic for assessing and treating patients, following the quality of glycemia in populations, determining trade-offs for developing algorithms for automated insulin delivery algorithms, and predicting long-term complications in diabetes.

Supplemental Material

sj-pdf-1-dst-10.1177_19322968221085273 – Supplemental material for A Glycemia Risk Index (GRI) of Hypoglycemia and Hyperglycemia for Continuous Glucose Monitoring Validated by Clinician Ratings

Supplemental material, sj-pdf-1-dst-10.1177_19322968221085273 for A Glycemia Risk Index (GRI) of Hypoglycemia and Hyperglycemia for Continuous Glucose Monitoring Validated by Clinician Ratings by David C. Klonoff, Jing Wang, David Rodbard, Michael A. Kohn, Chengdong Li, Dorian Liepmann, David Kerr, David Ahn, Anne L. Peters, Guillermo E. Umpierrez, Jane Jeffrie Seley, Nicole Y. Xu, Kevin T. Nguyen, Gregg Simonson, Michael S. D. Agus, Mohammed E. Al-Sofiani, Gustavo Armaiz-Pena, Timothy S. Bailey, Ananda Basu, Tadej Battelino, Sewagegn Yeshiwas Bekele, Pierre-Yves Benhamou, B. Wayne Bequette, Thomas Blevins, Marc D. Breton, Jessica R. Castle, James Geoffrey Chase, Kong Y. Chen, Pratik Choudhary, Mark A. Clements, Kelly L. Close, Curtiss B. Cook, Thomas Danne, Francis J. Doyle, Angela Drincic, Kathleen M. Dungan, Steven V. Edelman, Niels Ejskjaer, Juan C. Espinoza, G. Alexander Fleming, Gregory P. Forlenza, Guido Freckmann, Rodolfo J. Galindo, Ana Maria Gomez, Hanna A. Gutow, Lutz Heinemann, Irl B. Hirsch, Thanh D. Hoang, Roman Hovorka, Johan H. Jendle, Linong Ji, Shashank R. Joshi, Michael Joubert, Suneil K. Koliwad, Rayhan A. Lal, M. Cecilia Lansang, Wei-An (Andy) Lee, Lalantha Leelarathna, Lawrence A. Leiter, Marcus Lind, Michelle L. Litchman, Julia K. Mader, Katherine M. Mahoney, Boris Mankovsky, Umesh Masharani, Nestoras N. Mathioudakis, Alexander Mayorov, Jordan Messler, Joshua D. Miller, Viswanathan Mohan, James H. Nichols, Kirsten Nørgaard, David N. O’Neal, Francisco J. Pasquel, Athena Philis-Tsimikas, Thomas Pieber, Moshe Phillip, William H. Polonsky, Rodica Pop-Busui, Gerry Rayman, Eun-Jung Rhee, Steven J. Russell, Viral N. Shah, Jennifer L. Sherr, Koji Sode, Elias K. Spanakis, Deborah J. Wake, Kayo Waki, Amisha Wallia, Melissa E. Weinberg, Howard Wolpert, Eugene E. Wright, Mihail Zilbermint and Boris Kovatchev in Journal of Diabetes Science and Technology