Stress Testing of an Artificial Pancreas System With Pizza and Exercise Leads to Improvements in the System’s Fuzzy Logic Controller

Abstract

Background:

Under controlled conditions, the Dose Safety artificial pancreas (AP) system controller, which utilizes “fuzzy logic” (FL) methodology to calculate and deliver appropriate insulin dosages based on changes in blood glucose, successfully managed glycemic excursions. The aim of this study was to show whether stressing the system with pizza (high carbohydrate/high fat) meals and exercise would reveal deficits in the performance of the Dose Safety FL controller (FLC) and lead to improvements in the dosing matrix.

Methods:

Ten subjects with type 1 diabetes (T1D) were enrolled and participated in 30 studies (17 meal, 13 exercise) using 2 versions of the FLC. After conducting 13 studies with the first version (FLC v2.0), interim results were evaluated and the FLC insulin-dosing matrix was modified to create a new controller version (FLC v2.1) that was validated through regression testing using v2.0 CGM datasets prior to its use in clinical studies. The subsequent 17 studies were performed using FLC v2.1.

Results:

Use of FLC v2.1 vs FLC v2.0 in the pizza meal tests showed improvements in mean blood glucose (205 mg/dL vs 232 mg/dL, P = .04). FLC v2.1 versus FLC v2.0 in exercise tests showed improvements in mean blood glucose (146 mg/dL vs 201 mg/dL, P = .004), percentage time spent >180 mg/dL (19.3% vs 46.7%, P = .001), and percentage time spent 70-180 mg/dL (80.0% vs 53.3%, P = .002).

Conclusion:

Stress testing the AP system revealed deficits in the FLC performance, which led to adjustments to the dosing matrix followed by improved FLC performance when retested.

The pharmacodynamics of present-day insulins makes optimal glucose control difficult. Recent publications^1,2, review studies that use artificial pancreas (AP) systems to maintain some degree of glycemic control and facilitate simpler, more effective, disease management in individuals with type 1 diabetes (T1D). Recent studies on outpatients at home, ³ in hotels, ⁴ and at camps ⁵ have shown advantages of closed-loop treatment over pump or sensor-augmented pump therapy. One of these AP systems is the Dose Safety fuzzy logic controller (FLC).

Using the computing platform designed by University of California, Santa Barbara and Sansum Diabetes Research Institute, Dose Safety refined a closed-loop controller that utilizes “fuzzy logic” to analyze blood glucose data to calculate and deliver appropriate amounts of insulin.^6,7 The Dose Safety FLC incorporates the last 3 continuous glucose meter (CGM) values to recommend an insulin dose that is based on the slope and rate of change of blood glucose and the absolute glucose. ⁸ Unlike the other FLC developed by Phillip et al ⁵ the FLC uses only fuzzy logic technology with no other control system components.

Initialization of the FLC is done using the subject’s basal rate and carbohydrate ratio. The FLC also has a scaling factor which we call aggressiveness factor (AF) between 0 (most aggressive) and 10 (least aggressive) that is applied to the dose produced by postdosing matrix output; a lower AF means a higher dose in which more insulin is delivered by the controller for a given CGM signal. The AF is a predetermined linear scaling factor which is set at the start of each study and does not change during that study period.

We recently reported findings from a feasibility study that evaluated the FLC in the setting of bed rest, in a controlled environment, and assessing controller performance in response to carbohydrate-controlled meals without meal announcement or premeal bolus. ⁹ Seven of the 10 subjects who completed the study had mean blood glucose values ≤165 mg/dL and were within a specified target blood glucose range (70-200 mg/dL) for 76% of the 24-hour study period.

In this study, we stress tested the FLC performance in response to extreme glucose challenges under conditions that more reflect real-life use. A pizza meal represents one extreme due to its high carbohydrate (CHO) and fat content. Physical exercise represents the other extreme because exercise may require prior food intake and/or planned alteration of insulin dosing that limits spontaneity and requires subject intervention. Physical exercise can also increase the risk of delayed or nocturnal hypoglycemia. ¹⁰ We hypothesized that stress testing the FLC would identify deficits in performance when applied in real-life situations and that these deficits could be effectively addressed through modifications to the insulin-dosing matrix to improve overall glycemic results without increasing the incidence of hypoglycemia. This is a methods article to explain the process for modifying the controller to address correction of controller deficiencies.

Methods

The primary aim of this study was to demonstrate that stress testing the FLC during exercise and after a high-CHO high-fat meal (pizza) in a clinical research setting could lead to improvements in the AP system. In the study, subjects with T1D participated in 1 of the 2 study arms (exercise or pizza) for the respective glucose challenge, using version 2.0 of the controller (FLC v2.0). After approximately 50% of the studies were completed, an interim analysis was conducted to determine the efficacy of FLC v2.0 and modify the insulin-dosing matrix as needed. The remaining studies were conducted with the revised version of the controller (FLC v2.1) using the same AF as the previous studies. Changes in glycemic control comparing FLC v2.0 and FLC v2.1 were analyzed and reported here.

US Food and Drug Administration approval for the investigational device and institutional review board approval (Benaroya Research Institute) was obtained prior to study initiation, and all patients provided signed written consent for participation.

Subjects

We recruited 10 subjects with T1D through the Benaroya Research Institute Diabetes Clinical Research Program; all studies took place at the clinical research center (CRC). Subjects enrolled in either the pizza or the exercise arm of the study and agreed to participate in at least 3 assessments at the time of recruitment.

Inclusion criteria for study participation were duration of T1D for at least 1 year; age 18-50 years; use of insulin pump for at least 3 months; glycated hemoglobin (HbA1c) <9.0%; regular aerobic exercise (≥30 minutes 3 times/week); able and willing to give informed consent; and willing to participate in at least 3 study assessments (pizza or exercise) within 3 months. Key exclusion criteria were history of uncontrolled hypertension, chronic renal disease, anemia, cardiac arrhythmia, or known cardiovascular disease, or treatment with antidiabetes medications other than insulin.

AP System Components

The University of California, Santa Barbara APS platform was used for all studies. This APS system provides fully closed-loop glucose control. No meal or exercise announcements were used. All of the insulin used during each study was commanded by the FLC. This platform includes the FLC with computer, CGM system, and insulin pump. The FLC calculates the insulin dose based on the 3 previous CGM readings and delivers insulin every 5 minutes. Two versions of the FLC were used during the assessments: FLC v2.0 for the interim analysis phase and FLC v2.1 for the subsequent studies. Two CGM systems were used: the Seven Plus CGM system (Dexcom, San Diego, CA, USA) with FLC v2.0 and the Gen4 CMG system (Dexcom) with FLC v2.1. Both CGM systems measure glucose levels every 5 minutes and automatically transmit the data to the FLC. The OmniPod insulin pump (Insulet Corporation, Bedford, MA, USA) was used for all studies.

Procedures

Screening Visits

Subjects were screened for eligibility 2 days prior to their first study day. At the screening visit, subjects provided signed informed consent. Data from each subject’s personal insulin pump was downloaded to access total daily basal insulin dosages. Point of care HbA1c was measured in all subjects and pregnancy testing was performed in women of childbearing potential. Two CGM sensors in blinded mode were placed on each subject. Subjects were instructed to calibrate both sensors 2 hours after placement and at least every 12 hours until their visit for admission to the study

Exercise Studies

Exercise study subjects were instructed to eat their usual breakfast and dose insulin based on blood glucose values and CHO content before 8 am on study days. Subjects were instructed to arrive at the CRC at approximately 10 am and intravenous lines were placed for blood glucose measurement via Yellow Springs Instrument (YSI) every 30 minutes during the study period. The CGM sensors were set to the un-blinded mode and calibrated. At 11 am, subjects ate a lunch meal (60 gram CHO with protein and fat) and administered their usual insulin dose using their personal insulin pumps. The study insulin pump was placed at 12:30 pm and programmed to deliver the subject’s usual basal insulin rate. The AP system platform was initialized at approximately 2:00 pm using the subject’s 3-day mean daily basal insulin dose, CHO ratio, and assigned AF. All subjects started at AF 5 for their first exercise study. In earlier studies, this was considered to be the optimal AF setting to facilitate mean blood glucose of <160 mg/dL without hypoglycemia (<60 mg/dL). If the CGM indicated glucose levels between 70-100 mg/dL prior to exercise, a snack consisting of 15-20 gram CHO with fat and protein was given to achieve a blood glucose >100 mg/dL. At approximately 4 pm subjects performed approximately 5 minutes of warm-up exercise prior to the 30-minute exercise session (stationary bicycle) with a target of 70% of maximum heart rate. Subjects were not allowed to eat for 90 minutes postexercise unless they became hypoglycemic. Subjects ate a dinner meal consisting of 75 gram CHO followed by a small bedtime snack (20 gram CHO), which was a usual practice for most subjects. Participants were monitored on the AP system until 8 am the following morning.

High CHO/High Fat (Pizza) Meal Studies

Pizza study subjects were instructed to fast overnight (with the exception of hypoglycemia treatment) and arrive at the CRC at 7 am. Subjects were instructed to test and correct their blood glucose prior to arrival. (If the blood glucose was >250 mg/dL at arrival, the study visit was rescheduled.) An intravenous line was placed, CGM sensors were unblinded and calibrated, and the AP system platform was initialized as described above at approximately 9 am. The subjects were served a non-CHO breakfast. At 11 am, subjects ate the pizza meal (120 ± 5 gram CHO, 60 ± 5 gram fat). Subjects remained on the system until the study conclusion at 5 pm.

Statistical Analysis Methods

As prespecified in the protocol, individual studies were stopped if (1) the YSI glucose was ≥250 mg/dL for 4 hours or any occurrence of a YSI glucose >400 mg/dL or <60 mg/dL or (2) there was nonevaluable data due to component failure. (An error was made in basal dose calculation in 1 study; data from all other studies with evaluable data were included in the analysis.)

YSI blood glucose values were used for all outcome measures. The following measurements were used to assess controller performance comparing FLC v2.0 and FLC v2.1: mean blood glucose, mean maximum and minimum blood glucose, and percentage of time spent within/outside target blood glucose (<70 mg/dL, 70-180 mg/dL, 70-140 mg/dL, >180 mg/dL, >250 mg/dL). The pizza sessions were 10 hours, of which 8 hours were closed-loop. The exercise sessions were 20 hours, of which 18 were closed-loop. Hypothesis testing was done using a 2-sample single-tailed t-test assuming equal variance. P < .05 was considered statistically significant.

Results

Ten subjects were enrolled and participated in the pizza and/or exercise study. The characteristics of the subjects were as follows: 4 females and 6 males. The range, mean ± SD for age was 22-36 years, 27.3 ± 5.3 years, duration of diabetes 2-14 years, 9.5 ± 3.3 years, and hemoglobin A1c 5.7-8.4%, 7.0 ± 0.8%. The subjects participated in a total of 17 pizza and 13 exercise studies. Nine pizza studies were initiated with the FLC v2.0, 6 of which were terminated early either due to YSI blood glucose >250 mg/dL for greater than 4 hours or >400 mg/dL. The terminations prompted the modification in the FLC software from FLC v.2.0 to FLC v.2.1. Four exercise studies were initiated using FLC v2.0; 1 was terminated early due to YSI blood glucose >250 mg/dL for greater than 4 hours. Eight pizza studies were initiated using FLC v2.1; 1 was terminated early due to blood glucose <60 mg/dL 15 minutes before the end of the study. Nine exercise studies were initiated using FLC v2.1; 1 of the exercise studies was terminated early due to CGM failure and 1 was terminated early due to controller failure. One pizza study was terminated early due to hypoglycemia 15 minutes prior to complete was not included. These 3 studies were not included in the analysis. No study was terminated due to diabetic ketoacidosis (DKA). In summary, the analysis includes data from 16 pizza studies (9 with FLC v2.0, 7 with FLC v2.1) and 11 exercise studies (4 with FLC v2.0, 7 with FLC v2.1). Baseline glucose by YSI at the start of exercise studies was 142mg/dL, with a range of 111 to 185 mg/dL. For the pizza studies the baseline glucose by YSI at the start of the studies was 137 mg/dL, with a range of 111 to 185 mg/dL. The total daily basal insulin for the exercise subjects averaged 23.6 units/day, with a range of 12.4 to 40.0 units per day. The pizza study subjects averaged 25.1 units/day, with a range of 14.1 to 42.6 units per day.

Interim Analysis and FLC Modification

After 13 studies (9 pizza, 4 exercise) using the first version of the controller (FLC v 2.0), an interim data analysis was performed. Results from this analysis showed excessive hyperglycemia in both study arms; the majority (6/9) of the pizza studies were terminated early due to meeting predetermined stopping points for hyperglycemia. In addition, several instances of insulin dosing when CGM readings were below 100 mg/dL were observed, an undesired event. Based on our analysis of the FLC v2.0 data, the FLC insulin-dosing matrix was modified. Fifteen of the dosing cells in the dosing matrix were also increased by up to 170% for a given matrix cell value (Figure 1). Open-loop regression tests of FLC v2.1 were done on the FLC v 2.0 datasets to ensure that the FLC v2.1 changes to the dosing matrix produced the desired results. Following these changes, the remaining 17 studies were completed with FLC v 2.1.

Figure 1.

Fuzzy logic rules matrix. Blood glucose ranges are on the left. Note there is some overlap. Blood sugar rate VN is falling at −4 mg/dL/min, N is −2 mg/dL/min, Z if flat, P is rising at 2 mg/dL/min, VP is at 4 mg/dL/min. Blood sugar acceleration N is decelerating at −0.3 mg/dL/min², Z is near-linear, and P is accelerating at 0.3 mg/dL/min². Matrix modules were changed to address issue #1 (diagonal), #2 (cross-hatched), #3 (horizontal), and #4 (vertical) bars. The definitions for the matrix are based on the last 3 CGM values. Velocity is based on the first and the latest value. Acceleration utilizes all 3 values. The numbers in black represent the 2.0 version, and the numbers in blue are those changed to make the 2.1 version.

From the studies, 4 different characteristics were identified to improve the FLC v2.0 controller algorithm because there is no meal notification or premeal bolus a faster dosing response to the rise in blood glucose at the onset of a meal was desired (#1 on Figure 2), realizing that overall postmeal peak glucose was higher than desired (#2 on Figure 2 and Figure 3), recognizing that the overall blood glucose postmeal remained elevated (#3 on Figure 3), and inadequate dosing was found during the fasting periods of the exercise studies (#3 on Figure 2).

Figure 2.

Exercise study, BG average and SD by YSI, FLC v2.0 (n = 4) and v2.1 (n = 7).

Figure 3.

Pizza study, BG average and SD by YSI, FLC v2.0 (n = 9) and v2.1 (n = 7).

Subsequent to FLC v2.1 controller changes, the definition of the high (H) and normal (N) blood glucose inputs was revised to eliminate all dosing below 95 mg/dL (shaded area in the blood glucose membership column).

Additional changes, similar to those above, have been made in FLC v2.3, which will soon be clinically tested.

Glucose Control Following FLC Modification

Change from FLC v2.0 to FLC v2.1 showed improvements in glycemic control throughout the study time-period in both exercise and pizza studies. While there was no hypoglycemia during the exercise studies in either version of the controller, there was improvement in overall glycemic control when comparing FLC v2.0 to FLC v2.1 (Figure 2). Indeed, this was true both for the periods during and immediately following exercise and overnight. In the pizza studies, the improvement in glucose control is particularly evident starting 2 hours after consumption of the 120 gram CHO pizza meal (Figure 3). After the dosing algorithm adjustment in FLC v2.1, no pizza study was stopped due to hyperglycemia in contrast to 6 of 9 studies prior to the matrix changes. All of the exercise studies were completed.

FLC v2.1 versus FLC v2.0 in exercise tests showed improvements in mean blood glucose (146 mg/dL vs 201 mg/dL, P = .004), percentage time spent >180 mg/dL (19.3% vs 46.7%, P = .001), and percentage time spent 70-180 mg/dL (80.0% vs 53.3%, P = .002), >250 mg/dL (23.8% to 5.2%, P = .005) and maximum glucose of 244 versus 318 mg/dL, P = .008. No subject became hypoglycemic during exercise, postexercise, prior to dinner, or during the night.

Use of FLC v2.1 versus FLC v2.0 in the pizza meal tests showed improvements in mean blood glucose (205 mg/dL vs 232 mg/dL, P = .04), maximum glucose 276 mg/dL versus 340 mg/dL, and percentage time spent >250mg/dL 24.9% versus 48.0%. The mean average relative differences (MARDs) for the Seven Plus versus the Gen4 were 9.78 and 9.24, respectively, which are not different, P < .05. More detailed results for both the exercise and pizza studies can be found in the appendix (Figures A1 and A2).

Discussion

This report is intended to serve as a methods article showing how easily a matrix based FLC can be modified to address specific areas of concern. While the results from our feasibility studies were encouraging, we hypothesized that stress testing the FLC closed-loop system with pizza—a high CHO/high-fat meal—and exercise would more accurately reflect FLC performance in real-life conditions and, therefore, would reveal deficits in the dosing matrix and would allow us to determine if changes in 1 portion of the dosing matrix (to try to blunt the rise in glucose caused by pizza) would negatively affect our ability to handle unannounced exercise. Although the maximum glucose post–pizza meal was improved, more testing will be required to determine if a premeal bolus is required for such a large meal.

Although we recognize that individuals with T1D are unlikely to regularly consume 120 grams of CHO at 1 meal, we deliberately selected this extreme CHO challenge—the equivalent to one half of a large pizza—to maximally stress the AP system. The other extreme challenge is exercise; people with T1D typically must anticipate exercise and adjust food and/or insulin prior to activity. In this study, individuals exercised without consuming a scheduled snack and without adjusting insulin dosing in anticipation of exercise. Moreover, the subjects did not eat again until 1.5 hours after finishing the exercise, when they ate dinner. Subjects were permitted to eat a bedtime snack, as was their usual practice at home. It is striking that even with these restrictions, there was no immediate, delayed, or nocturnal hypoglycemia with the FLC controlling insulin delivery. These results illustrate the potential of the FLC to allow individuals with T1D to exercise spontaneously. The snack at bedtime may have influenced their nocturnal hypoglycemia but was their usual practice on days with and without exercise.

We recognize that switching from the Seven Plus to the Gen4 CGM is a limitation to our study. The overall MARDs were 9.78 and 9.24, respectively, which may influence the improvements but doesn’t explain the degree of improvement. Another concern is the number of uncompleted studies. Most of the uncompleted studies in version 2.0 (6 out of 9) were in the pizza studies for prolonged hyperglycemia and precipitated the conversion to FLC v2.1. Of the 3 other incomplete studies, 1 was due to controller failure, 1 was due to sensor failure, and 1 was secondary to hypoglycemia. None was due to hyperglycemia after converting to the 2.1 version. Another limitation of this study is the lack of a control group.

The key finding from this study is that stress testing highlights deficits in the AP system and leads to improvements. As demonstrated, significant glycemic improvements were seen between FLC v2.0 and FLC v2.1 for most endpoints, regardless of the time period, for both pizza and exercise studies.

Development of an effective AP system is an iterative process. Findings from our study are valuable both in identifying the need for stress testing in the AP system process and in demonstrating that stress testing leads to improvements in AP system performance.