Double Dummy Design for Blinding Studies With Automated Insulin Delivery Systems: A Proof of Concept Trial

Abstract

Background:

Comparative assessment of therapeutic technologies is often biased due to the inability to blind interventions, especially when therapies differ in form or dosing. While double-dummy design, where participants receive both an active treatment and a matched placebo to maintain blinding, is well established in pharmacological trials, its applicability for medical devices requiring user interaction, such as automated insulin delivery (AID) systems is challenging.

Methods:

We present the methodology by which two AID systems are used in a double-dummy, blinded, randomized trial, one system providing insulin therapy and the other, a diluent.

Outcomes and conclusion:

The study demonstrates the feasibility, of comparing 2 AID systems, without operartor bias.

Comparative assessment of technologies used for therapeutics are fraught with challenges primarily due to the inability of the therapies to be blinded. This introduces biases due to the preferences, sponsorships, either intentional or subliminal. In pharmacological studies, this has been addressed with the introduction of the double dummy study design, ¹ when the comparator interventions are of different nature (eg, tablets vs injectables) or dosing intervals (eg, daily vs weekly injections). In these studies, identical placebos with similar physical appearances, but without the active component are used as controls, to each of the interventions (eg, tablets and injections). Participants are then randomized in a blinded way to the 2 components assessed, one that consists of the active component and the other one a control. The study design can thereafter include a crossover stage. This design is limited when regards to medical devices, where repeated interactions with the device are required for achieving outcomes. Automated insulin delivery (AID) systems are a notable example as these devices are reactive to continuous glucose measurement aimed to achieved glycemic control which is affected by user interactions. In addition, not all AID systems are driven by similar sensors, and no continuous glucose monitoring (CGM) system to date can be linked to more than one system. Therefore, it is difficult to create placebo devices that can effectively blind both participants and investigators without compromising the study’s integrity or the participants’ safety. To the best of our knowledge, no studies have been published to date, comparing insulin delivery systems utilizing a double-dummy design to address user and investigator biases.

In this report we describe the methodology and outcomes of a feasibility proof of concept double-dummy blinded randomized trial comparing 2 different AID systems.

Study was registered within the Israeli Ministry of Health number MOH_2023-04-09_012516.

Aim: The purpose of this study was to assess the feasibility to compare glycemic control of 2 AID systems in adult subjects with type 1 diabetes in a blinded manner.

Method: This is a single-center, randomized, double-dummy controlled trial comparing 2 AID systems in adult subjects with type 1 diabetes. The systems were MiniMed 780G Insulin Pump with Guardian Sensor 4 Glucose Sensor referred to as “MM780G” and Tandem t: slim X2 with Control-IQ Technology with Dexcom G6 CGM referred to as “Control-IQ.”

The study design included 3 phases. A 5-week at home run-in period, followed by two 6-day protected home environment phases, separated by a 2-week washout period. During the 5-week run-in period, all participants were familiarized with the Control-IQ device and this period was used to optimize pump settings. During the last 14 days of run-in period, all participants were switched to MM780G. The last 14 days on Control-IQ device during the run-in period and the last 14 days on MM780G were used for comparison of the unblinded period. The study flow is presented in Supplemental Figure S1.

Randomization

During these 5 weeks of run-in, randomization by draw defined the device sequence. The sequence was known to only one designated coordinator, while the rest of the team and subjects remained blinded to the AID system allocation.

“Protected Home” Phase 1

Day 1

The subjects were connected to 2 AID systems, simultaneously. The pre-programmed and optimized Control-IQ system connected to a Dexcom G6 CGM and MM780G connected to a guardian 4 sensor. Each system was connected to the body with the respective designated infusion set. One system was filled with an insulin diluent and the other with insulin according to the randomization scheme. Both the participant and the treating personnel were blinded as to which device was administering the insulin. Therefore, all interactions with the devices, such as meal announcements and corrections were performed simultaneously by the user in both systems according to the respected CGM.

Days 1-4

The systems were used as intended with pre meal boluses provided within 0 to 15 minutes of meal initiation, in a consistent manner. All meals (time and content) and the routine exercises were predefined, to ensure the ability to be repeated at phase 2 of the study.

Day 5

On this day, all meals consisted of up to 80 g of carbohydrate and were unannounced, and no correction boluses were initiated by the users post meals.

Day 6

Morning exercise consisted of 45 minutes of aerobic exercise at the participant’s usual intensity with temp target/Control-IQ sport activated 1 hour prior to the activity.

Wash Out Period

Up to 3 weeks between Phase 1 and phase 2. Participants were provided with a new MM780G device.

“Protected Home” Phase 2

Day 1

The subjects were reconnected to the 2 AID systems. The preprogrammed and optimized Control-IQ system connected to Dexcom G6 CGM and the MM780 connected to Guardian 4 sensor. According to the randomization scheme, the system that was filled with a diluent in phase 1 was switched to insulin and vice versa. The insulin/diluent cartridges were prepared by the unblinded coordinator, while both the participant and the treating personnel remained blinded to the active device allocation. As in phase 1, all subject initiated interactions with the pump were performed simultaneously on both systems according to the respective CGM.

Day 1-6

Followed identical schedule, meals, and routine as in phase 1 (described above).

Pump Settings

MM780G:

• The pump used SmartGuard Auto Mode turned on.

• The automatic target was set to 100 mg/dL.

• Active insulin time was set at 2 hours.

• Bolus rate was set to Quick rate with a minimum bolus increments of 0.025 u.

Control-IQ:

• User’s body weight in kg.

• Average daily insulin use in units.

• Basal rates as optimized during the run-in period.

Study Approval and Informed Consent

The study was approved by Sheba Medical Center institution ethical review board. All participants signed an informed consent to participate in the study. Study was registered within the Israeli Ministry of Health number MOH_2023-04-09_012516.

Outcomes

Participants: Four participants were assessed with this protocol. Ages were 33, 38, 29, and 50 years, 2 were female and the duration of diabetes span from 7 to 31 years. Mean HbA1c was 7.57%.

Run in Unblinded Period

Comparing data of 14 days at the end of the run-in period showed good glycemic control with both systems with a tendency of lower average glycemic (SD) levels with MM780G versus Control-IQ 150.1 ± 8.4 versus 157.1 ± 17.3 mg/dL, respectively, corresponding to glucose management indicator (GMI) of 6.9 ± 0.2 versus 7.1 ± 0.4 (Figure 1). TIR^70-180, TBR⁷⁰, TBR⁵⁴ in % (SD) were 74.8 ± 6.6, 1.1 ± 0.5, and 0.3 ± 0.2 versus 70.7 ± 10.3, 1.1 ± 0.5 and 0.2 ± 0.2 with MM780G versus Control-IQ, respectively.

Figure 1.

Participants combined CGM data during run in.

System and CGM use were high in both systems. Total daily insulin was similar at 47 units/day though number of meals (5.8 ± 4 vs 6.3 ± 2.4) and amount of total daily carbohydrate entered (145.6 ± 99.2 vs 191.8 ± 90.8 g/24 h) were numerically higher with the Control-IQ system. No statistical comparison were applied as these were exploratory data and not powered for outcomes.

Blinded Period

Continuous glucose monitoring data during the blinded 2 phases were collected simultaneously by the 2 sensors and found to be comparable though not identical (Supplemental Figure S2).

The glycemic outcomes were analyzed with data from both sensors. Control of glycemia was better when the CGM data derived was from the sensor driving insulin delivery by the system’s algorithm. When the MM780G was driving the insulin infusion, mean ± SD glucose was 138.9 ± 10.1 with TIR^70-180 of 83.7% ± 8.2% as recorded by the GS4, while at the same time, the Dexcom CGM recorded a mean glucose of 142.4 ± 8.3 and a TIR of 78.6% ± 5.4%. The same metrics when the control-IQ was active with insulin yielded a mean glucose of 147.7 ± 9.3 mg/dL and a TIR of 78.8 ± 6.6 recorded by the Dexcom, and 150.1 ± 15.6mg/dL and 77.1% ± 6.1% recorded by the G4S. TAR^>250 was numerically higher with Control-IQ in both phases, as measured by both CGMs (Table 1).

Table 1.

Combined CGM Metrics by the Active System Used During Blinded Phases According to the Sensor Used A-G4S and B-Dexcom.

CGM Metrics During Blinded Unannounced Meal

Comparison of per participant, 5 hours CGM data following unannounced meals versus announced meals (identical content) with the 2 systems is presented in Figure 2.

Figure 2.

Participant’s combined stacked CGM data in the 5 hours post meal announcement (blue) and unannounced meals (red), by either the MM780G system (left panels) or Control-IQ (right panels). Data are presented for both Dexcom and G4S sensors.

Of note, there are individual differences, as participants 1 and 3 show similar glycemia when announcing or not announcing using the MM780G system, while the unannounced meal with Control-IQ resulted in higher glucose excursion. However, in participants 2 and 4, there were higher postprandial excursions following unannounced meals in both systems.

Discussion

The aim of the study was to assess the feasibility of a novel approach in studying automated insulin devices to eliminate biases related to lack of blinding. This small sample, which is not powered to detect efficacy outcomes, demonstrated that a double dummy blinded randomized study, such as utilized in trials comparing physically different pharmacological interventions, ¹ can be performed successfully with medical devices. However, there are several caveats that require attention when designing such studies:

1. Study duration and participating subjects—The use of 2 systems simultaneously requires attention by the user and increases user’s burden. Doubling wearables, such as insulin pumps infusion sets and sensors, doubling the bolus process, for meals and corrections and the need to undertake corrective actions on both systems is difficult and demanding. For example, in case of an infusion site failure, the need to change both sets is not trivial. These challenges limit the study duration of the double and may limit the study to highly motivated selected individuals. In this study the limitation was for 2 phases of 6 days.

2. Study location—as this is a complex operation for both the user and the study staff, we opted for a protected home environment, used in previous studies ² that is an environment under supervision that mimics a home environment. Though it does impose deviation from daily life, it affects both therapies similarly.

3. Study design- as the duration of the study is limited, it is pertinent to compare similar periods in behavior, both of meals and exercise. In this study, there was meticulous attention to provide the same routine, individualized to each participant in terms of meal timing and constituents, exercise schedule and intensity, so the same routine will be followed in both phases. Conducting the study in a protected home environment provides an additional advantage for this strict study design.

4. Familiarity with both devices- some users might be familiar with one or the other device. Ample adjustment for learning the new system(s), as was done in the run-in phase of this study, is to be provided.

5. Optimizing the algorithm requirement to optimize the outcomes of each system. This was incorporated within the study design. Optimization of the control-IQ system before the blinded period as well as the requirement that at least one week of insulin delivery by the pump will be provided to the MM780G users was verified in this study. This should be adjusted to each system studied and complicates the run-in period.

6. Control used—The control to the insulin used was the original diluent of the insulin used. It’s important that there will be no distinction in the physical properties of the solutions, mainly smell. As the vials were labeled, a coordinator, uninvolved in the study, maintained the insulin pump cartridge refilling according to the randomization schedule. This can be eliminated if vials of insulin and diluents were labeled without reference to the content.

7. Outcome measures—Insulin delivery by the algorithm is calculated according to CGM signals provided by the sensors. There are well known differences in paired CGM reading from 2 sensors either intra-brand^3,4 and also between sensors of different manufacturers.^5-7 In this study data is presented from both sensors, those driving the insulin and those driving the placebo.

From an outcome perspective, this study is limited in conclusions especially with regards to overall performance between the systems. The small number of participants and the relatively short duration of the study were not powered to demonstrate statistical differences in glucose metrics. It is also noted that the study was conducted in a highly specialized center in technology for type 1 diabetes therapy as well as the participants are well experienced users. What appears is that each CGM records better results when their corresponding pump is performing the control (Table 1), which is explainable as the algorithm is attempting to adjust insulin delivery by data from the CGM it is connected to. We also show that overall sensors qualitatively performed in a similar fashion with no clear tendency to over or under read glucose levels by one brand or other (Supplemental Figure S2). There is a tendency for higher time spent in the high range when using Control-IQ during the study (Table 1) as assessed by both sensors, which could be due to the differences of the algorithm approaches to corrections, which is hourly in Control-IQ and every 5 minutes with MM780G. These differences may also play a role in the management of unannounced meals. Two patients demonstrated a significant better management of unannounced meal with MM780G in comparison to Control-IQ, while no clear differences were noted in the other 2 participants (Figure 2).

The blinding of the therapy is important when assessing outcomes with devices in treatment of diabetes in order to overcome multiple biases of selected reporting, participant selection, and site experience, among others. We found one study in the literature using partial blinding (“shadow mode”) to overcome these biases and actually reported findings that were not previously reported with a DIYAPS. ⁸

In conclusion, this limited study provides insights of conducting a blinded study with AID systems, showing feasibility and discussing the caveats and challenges in conducting such prospective double dummy design randomized control study.

Supplemental Material

sj-docx-1-dst-10.1177_19322968251409820 – Supplemental material for Double Dummy Design for Blinding Studies With Automated Insulin Delivery Systems: A Proof of Concept Trial

Supplemental material, sj-docx-1-dst-10.1177_19322968251409820 for Double Dummy Design for Blinding Studies With Automated Insulin Delivery Systems: A Proof of Concept Trial by Amir Tirosh, Andrea Benedetti, Nama Peltz-Sinvani, Maya Laron-Hirsh, Yael Cohen, Amna Jabarin, Benny Grosman and Ohad Cohen in Journal of Diabetes Science and Technology