Abstract
For individuals with type 1 diabetes (T1D), daily insulin intake is vital to regulate the glucose levels and reduce the risk of diabetes-related complications. The recent advances in continuous glucose monitoring (CGM) systems, continuous subcutaneous insulin infusion pumps, and control algorithms for closing the loop between CGM devices and pumps have moved forward the realization of an artificial pancreas (AP). In a typical scenario, the individual with T1D is using both a CGM system and a pump. The CGM is measuring the glucose concentration and provides input to a control algorithm running on a portable device (smartphone/tablet/laptop). The algorithm optimizes the insulin infusion, and in some cases the appropriate glucagon infusion, toward improved, safe, and prompt glucose regulation. Backbone of the AP is the control algorithm. A broad spectrum of control strategies has been proposed: proportional–integral–derivative (PID), model predictive control (MPC), fuzzy logic (FL), and very recently reinforcement learning (RL).1,2 Comprehensive reviews of the PID, MPC, and FL control strategies within the AP framework can be found in Doyle et al, 3 Kudva et al, 4 and Peyser et al. 5 However, it has to be noted that the majority of the proposed strategies rely on glucose measurements only, although it is well known that glucose is affected by a plethora of parameters related to lifestyle (eg, eating habits, physical activity), patient-specific characteristics (eg, body mass index, age) and metabolic status (eg, insulin sensitivity, other diseases, medication, stress levels).
Very recently the integration of information related to physical activity 6 and insulin sensitivity 7 has started being investigated, while for the effects of meals in the postprandial glucose regulation two different approaches have already been proposed. In the first, the user is manually announcing the meal (time and carbohydrate amount) to initiate the infusion of the premeal insulin bolus dose, while in the second an algorithm detects the meal and estimates its size.8,9 The major shortcoming of the manual meal announcement is the proven inaccuracy in carbohydrate counting of even trained individuals with T1D.10-12 In the automatic meal announcement, the developed meal detection algorithm analyses the CGM data before they enter an MPC system. The in silico results indicate that the algorithm was able to detect the meals 30-45 minutes after their intake. Although this delay results in delayed insulin boluses and slightly higher postprandial glucose concentration, the average glucose is comparable to the case of manual meal announcement. In summary, both methods have certain limitations either because of erroneous carbohydrate estimation or due to delays in detecting the meal onset. The majority of the already proposed control algorithms have been evaluated with respect to their ability to reject the meal disturbance either in silico—by introducing uncertainties in the meal protocol—or in clinical setup. However, only a limited number of studies were focused on the postprandial glucose control. According to Chase et al, 13 in the case of control algorithms without meal announcement, the American Diabetes Association’s goal to have postprandial glucose levels below 180 mg/dl is not met, 14 indicating that meal announcement is required with the currently available insulin types.
Recently, the introduction of adaptive control strategies has been proposed to address the challenges related to inter- and intrapatient variability, and uncertainties in disturbances, for example, meal and physical activity.2,6 Although real-time learning permits the continuous adjustment and personalization of insulin therapy that overcomes most disturbances, the announcement of meals may still add robustness against patient variability and uncertainties.
To address the limitations related to the meal announcement approach, the GoCARB system is introduced. GoCARB is a novel system based on computer vision and smartphone technologies and is designed to support individuals with T1D to count the meal’s carbohydrate content with an error less than 20 grams by just using 2 meal images. The prototype provides output in less than 15 seconds, while minimum user interaction is required. Scope of this article is to present GoCARB within the AP framework (GoCARB-AP) in an in silico environment. The procedure is conducted as a preparatory step of the clinical evaluation and involves individuals with T1D under sensor-augmented pump therapy using GoCARB for CHO estimation, initially in open and then in closed-loop approach. To this end, the developed prototype provides input to an “in-house” adaptive control algorithm based on RL. 2 It has to be noted that GoCARB is independent of the applied control strategy. The present GoCARB-AP implementation is evaluated in a number of meal scenarios. The structure of the article is as follows. In the next section the integrated system is presented followed by the study protocol. Then, the results are presented and discussed. Concluding remarks and future research directions complete the present research study.
System Outline
The combined use of the GoCARB system with a control algorithm for closing the loop between a CGM and a pump is presented in Figure 1, while the major involved components, namely the GoCARB system and the RL-based control algorithm, are briefly presented in the next paragraphs.
Conceptual diagram showing the combined use of the GoCARB system with a control algorithm. The control algorithm closes the loop between the CGM and the insulin infusion pump. The GoCARB takes as input images of meals and provides estimation of the carbohydrate content of the meals.
GoCARB System
GoCARB is a smartphone based system designed to provide automatic, accurate and near real-time carbohydrate (CHO) estimation for nonpacked foods to support T1D patients with prandial insulin dose estimation.15-18 The application requires as input a pair of images of the upcoming meal with a credit card-sized reference object placed next to it. The first image is acquired horizontally above the dish and the second at 20-30 degrees from the vertical axis crossing the center of the dish. Simple tools integrated in the graphical user interface of the application guide the user in choosing the optimal angles based on the smartphone’s built-in motion sensors. The optimal shooting distance is the smallest possible that permits a full view of the dish together with the reference object, which is usually around 30-40 cm. As soon as both images are taken, they are transmitted to a dedicated server via Wi-Fi or the mobile network, where a series of computer vision steps takes place. First, plate detection is performed and then the different food items on the plate are automatically segmented and recognized based on color and texture information. Food volume estimation follows, which relies on the reconstruction of the meal’s 3D shape by using both images and stereo vision techniques. The real dimensions of the produced shape are provided by the reference card, while the segmentation result is used to separate the volumes for the different foods. Finally, the CHO content is estimated by considering the recognized food types, the corresponding volumes, and information coming from the USDA nutritional database. The result is then transmitted back to the smartphone and displayed to the user. The current version of the system is a prototype that considers 9 broad food classes found in common central European meals. The system’s mean absolute percentage error was 10 ± 12% (or mean absolute error of 6 ± 8 CHO grams) in a laboratory setup 15 and 28 ± 20.5% (or mean absolute error of 13.16 ± 10.16 CHO grams) when being used by T1D patients in a preclinical study. 19
Control Algorithm
An adaptive model-free control strategy based on the Actor-Critic (AC) algorithm has been designed, developed, and in silico evaluated to compensate for the inter- and intraindividual variability of the diabetic population and the associated uncertainties. 2 The principle of the AC algorithm relies on RL and approximates an optimal control strategy through real-time learning in an iterative manner. It evaluates the current control policy (critic) and updates it (actor), according to the patient-specific characteristics yielding an optimized insulin infusion policy. The CGM gives glucose measurements as input to the AC algorithm. The resulting insulin infusion policy consists of the basal rate (BR) and bolus doses, both delivered by an insulin pump. AC optimizes the daily BR and insulin:carbohydrate (IC) ratio for each patient based on his/her measured glucose profile. The IC ratio is used for the calculation of the bolus dose according to the announced CHO content of the upcoming meal. For the automatic and personalized tuning of the AC, a method based on the estimation of information transfer (IT) from insulin to glucose signals is used. Insulin-to-glucose IT is linked to patient-specific characteristics related to total daily insulin needs and insulin sensitivity. 20 The AC needs minimal computational time and its use does not require interventions either by physicians or engineers.
Study Protocol
A series of computer simulations were designed to assess the combined use of GoCARB together with the AC control algorithm on T1D patients. Scope of the study is to prove whether the use of GoCARB by T1D patients on open loop (OL) or closed loop (CL) could improve their postprandial glucose control. To this end, we used a population of 10 adult T1D patients provided by the Food and Drug Administration accepted University of Virginia T1D simulator. 21 This group of patients was considered to be either on OL, simulating the standard treatment in which BR and IC ratio were defined by the treating physician, or on CL control based on the AC algorithm. For each case, the patients were either estimating their CHO intake by themselves or alternatively by using the GoCARB system, leading to a number of 4 different scenarios.
To achieve a realistic simulation of the performance in CHO counting for both patients and GoCARB, we used the results of a relevant preclinical study we recently conducted. 19 The scope of the study was to test the accuracy of the GoCARB system in comparison to the patients’ estimations. To this end, 18 T1D adult patients were asked to give estimates about the CHO content of 6 different dishes and then use the GoCARB application on them. Among the participants, 16 were from outpatient’s clinic of the Bern University Hospital “Inselspital” and trained in CHO counting, while the remaining 2 were of unknown experience in CHO counting. Since the results of the study indicated high interpatient variation, we decided to split the patients into 3 groups—good, moderate, and bad—according to their CHO estimation skills and repeat the aforementioned 4 scenarios for each group leading to a final number of 12 simulations. In this way, the performance of the proposed system will also be assessed with respect to the patient’s CHO counting skills. Good estimators were the ones with a mean absolute percentage error up to 30%, moderate estimators were those from the rest with errors less than or equal to 70%, and the rest were considered as bad estimators. The aforementioned errors were mostly equally distributed around zero, corresponding to either over- or underestimation.
In all the simulations the patients received 3 meals per day: breakfast, lunch, and dinner.
Breakfast: The GoCARB system is designed for meals served in a dish. Therefore, for the breakfast a random CHO content defined by a uniform distribution in the range of 30-60 grams was used. 22 The error in CHO counting for the breakfast was set to zero.
Lunch: For lunch the CHO content was defined by randomly choosing 1 of the meals of the preclinical study (16-125 grams). The considered error in CHO estimation was exactly the one recorded in the preclinical study both for the GoCARB and for the patients’ own estimates.
Dinner: The same as for lunch.
The timing for the meals was randomly defined by a uniform distribution in the range of 7:00-11:00, 13:00-16:00, and 20:00-22:00 for breakfast, lunch, and dinner, respectively. The total duration of each simulation was 14 days. In the case of CL, for the first 4 days OL glucose control was applied, followed by a 5-day training phase using the AC algorithm, while the rest were used for evaluation.
Results and Discussion
As mentioned before, the effects of GoCARB in postprandial response and in the individual’s glucose control were investigated using a number of computer simulations.
Postprandial Response
The following parameters were defined to describe the glucose profile in the postprandial period from the time of food intake up to 4 hours following the meal: the area under the curve (AUC) of the glucose profile, the peak glucose (PG), and the percentage of the time the glucose profile is between 70 and 180 mg/dl (P70_180) or exceeds 180 mg/dl (P180). The values of the main parameters that describe the glucose profile in the postprandial period are presented in Table 1.
Mean Values of the Parameters That Describe the Glucose Profile in the Postprandial Period for Each Simulation
AUC, area under the curve of the glucose profile; PG, peak glucose; P70_180, percentage of the time the glucose profile is between 70 and 180 mg/dl; P180, percentage of the time the glucose profile exceeds 180 mg/dl.
Table 1 shows that the use of GoCARB increases the time spent in target range (70 ≤ glucose ≤ 180 mg/dl), reduces the PG value, and minimizes the percentage of postprandial glucose levels above 180 mg/dl. Furthermore, it decreases the 4-hour AUC for both OL (simulations 1b, 3b) and CL (simulations 2b, 4b) approaches. In the case of simulation 5a, bad estimators in OL without GoCARB, for the 50% of the virtual patients (patients 1, 2, 8-10) the glucose level decreased to zero and remained constant until the end of the simulation. The rest (patients 3-7) had extreme postprandial hypoglycemic events (an example is presented in Figure 2). Thus, the 4-hour AUC is substantially lower in simulation 5a than in simulation 5b, where the use of GoCARB has diminished all the above mentioned extreme situations. The same is observed in simulations 6a and 6b. It has to be noted that for simulations 5a and 6a, the results presented on the Table 1 do not take into consideration the virtual patients 1, 2, and 8-10. In terms of postprandial control, the contribution of GoCARB is mostly significant in individuals with T1D with a mean absolute percentage error in CHO counting greater than 30% (moderate and bad estimators). The combined use of GoCARB and control algorithm seems to be more important for bad estimators (simulation 6b), since it achieves higher time spent in the target range, eliminating the number of glucose values above 180 mg/dl.
Example of extreme postprandial hypoglycemic event for bad estimators. The postprandial glucose level is shown from the beginning of the meal at 10
Glucose Control
The evaluation criteria used to assess the performance of combined use of the GoCARB with the AC algorithm were the time spent in normoglycemia (70 ≤ glucose ≤ 180 mg/dl), mild hypoglycemia (50 ≤ glucose < 70 mg/dl), severe hypoglycemia (glucose < 50 mg/dl), mild hyperglycemia (180 < glucose ≤ 300 mg/dl), and severe hyperglycemia (glucose > 300 mg/dl). The results are summarized in Table 2. Furthermore, the Low Blood Glucose Index (LBGI) 23 was calculated.
Percentage of the Time Spent in Normoglycemia (70 ≤ glucose ≤ 180 mg/dl), Mild Hypoglycemia (50 ≤ glucose < 70 mg/dl), Severe Hypoglycemia (glucose < 50 mg/dl), Mild Hyperglycemia (180 < glucose ≤ 300 mg/dl), and Severe Hyperglycemia (glucose > 300 mg/dl), on Average, for Each Simulation.
When GoCARB was applied, the glucose control was improved by reducing the time spent in hypo- and hyperglycemic ranges and increasing the time spent in target range. According to Table 2, there is generally an improvement of the glucose regulation when the AC control algorithm is applied regardless of the CHO estimation and the use of GoCARB. The “in-house” developed RL-based controller is able to optimize the daily insulin infusion through learning of important patient-specific characteristics and deal with high uncertainties. However, the most efficient regulation is obtained in the case of the combined use of AC and GoCARB. This can be further seen in Figure 3, which presents the daily evolution of the LBGI for the total duration of the study and for the case of individuals with T1D with a mean absolute percentage error in CHO counting greater than 70% (bad estimators). Without GoCARB, both the OL and CL approaches fail to control glucose with some of the virtual patients experiencing glucose concentrations equal to zero until the end of the simulation and the rest extreme hypoglycemic events (Figure 3a). By using GoCARB, glucose control is achieved (Figure 3b). Furthermore, the parallel use of the AC algorithm further reduced the risk of hypoglycemic events (in both training and evaluation phases).
Daily evolution of the LBGI in the case of individuals with T1D and mean absolute percentage error in CHO counting greater than 70%, when the OL approach (blue) or CL approach (orange) is used. The CHO content of the meals is estimated (a) without the GoCARB and (b) with the GoCARB. The results are presented for the total duration of both the OL and CL (training phase, evaluation phase) simulations.
Conclusions
In the present study a novel approach for glucose management is presented. The approach combines the GoCARB system, which supports individuals with T1D to count a meal’s CHO content, with an adaptive control algorithm within the AP framework. The approach is assessed in a series of computer simulations, prior to conducting a clinical evaluation that involves individuals with T1D. In summary, the results show improved control of the postprandial response and glucose profiles when the GoCARB is used, as there is increase in the time spent in target range, reduction in the time spent in hyperglycemia, and reduction in the peak glucose. In the case of OL, the use of GoCARB substantially improves the glucose regulation. The use of the AC control algorithm generally improves the glucose regulation. The most efficient regulation is obtained when there is combined use of the AC control algorithm and the GoCARB. However, there is still a need for further algorithmic optimization, integration, and clinical evaluation of GoCARB both as a standalone system and within the framework of the AP. The GoCARB prototype needs to be extended to additional food classes to deal with issues related to the diversity of eating habits. The use of adaptive learning approaches and involvement of users from different countries might be an important step toward this direction. Furthermore, the GoCARB’s CHO estimation can be fed to a bolus calculator to estimate the amount of insulin needed to compensate the effects of the CHO contained in a meal. Finally, it has to be noted that the GoCARB system is absolutely independent from the type of the used control algorithm, while after minor technical modifications, it could be applicable to individuals with type 2 diabetes as well.
Footnotes
Abbreviations
AC, Actor-Critic; AP, artificial pancreas; AUC, area under the curve; BR, basal rate; CGM, continuous glucose monitoring; CHO, carbohydrate; CL, closed loop; FL, fuzzy logic; GoCARB-AP, GoCARB within the AP framework; IC, insulin:carbohydrate; IT, information transfer; LBGI, Low Blood Glucose Index; MPC, model predictive control; OL, open loop; PG, peak glucose; PID, proportional–integral–derivative; P180, percentage of time the glucose profile exceeds 180 mg/dl; P70_180, percentage of time the glucose profile is between 70 and 180 mg/dl; RL, reinforcement learning; T1D, type 1 diabetes.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded in part by the Bern University Hospital “Inselspital” and the European Union Seventh Framework Programme (FP7-PEOPLE-2011-IAPP) under grant agreement 286408 (
).