Abstract
Commercial automated insulin delivery (AID) systems are usually assessed based on clinical outcomes, ignoring uptake. A qualitative study evaluated user experiences when switching to currently available commercial AID. Interview feedback was coded on key themes including the adoption experience with regards to quality of life, clinical outcomes, and users’ expectations. Most felt their learning curve was easy. Most saw reduced hypoglycemia and increased time in range, although there were outliers. Many mentioned post-meal hyperglycemia as an improvement area for commercial AID. Users with one particular continuous glucose monitor (CGM) type reported sleep disruption. Companies should consider real-world user feedback with regards to improving training materials for new users with less CGM experience and by improving target flexibility and postprandial algorithm performance, plus reducing manual interventions required by users.
Introduction
Commercially manufactured automated insulin delivery (AID) systems are now available in multiple countries worldwide. While there is growing literature around the clinical outcomes of AID systems, there is less attention to the learning curve, training, and uptake of commercial systems. 1 Switching from manual to AID systems can be challenging. 2 This study seeks to gain feedback from users in the real world about their experience switching from manual to commercial AID and their use of these systems in everyday life.
Methods
Key themes to solicit user feedback were developed in collaboration with Amy Tenderich (as this research was commissioned for the Summer 2020 DiabetesMine D-Data ExChange event) regarding:
Training and switching from previous care to AID.
Impact on time in range, including hypoglycemia.
Quality of life impacts, including sleep.
Troubleshooting and system upkeep requirements.
Expectations of commercial AID and whether these were met.
From this list of key themes, a list of semi-structured interview questions were developed for use in interviews with study participants (see Supplemental Appendix S1 for the list of study questions).
Keyword searches on social media (Instagram and Twitter) yielded a list of individuals posting publicly about their experience with commercial AID (Medtronic 670G; Tandem Control-IQ). A mix of potential participants was identified based on the researcher-perceived assessment of the length of time of use of the commercial AID. Users with a mix of positive or negative experience, including participants who had switched away from the commercial AID, were sought. Another system became available in the United Kingdom during the study, and an individual with early experience using the CamAPS FX (CamDiab) system was also sought.
After this list was created, outreach was sent via private messages to the identified users, inviting them to participate in the study, and sharing a simple study overview and informed consent form. A $50 honorarium, sent via PayPal or Amazon gift card, was offered for participation. Of the nine potential participants identified, seven chose to participate.
Each interview took place over the phone. Transcription-level notes were recorded for each call. After each interview, these notes were immediately reviewed and a summary was created for each participant, noting high-level positive feedback, frustrations or challenges, and any other feedback unique to the participant’s experience.
After interviews were complete, the individual participants’ transcription-level and summarized notes were reviewed (See Supplemental Appendix S2 for a written summary of each participant, and Table 1 for summary and demographics). A simple three-variable score was created for the key themes of the study (see Figure 1). A green checkmark indicates that the participant saw improvements or did not experience or comment on any issues, a black checkmark indicates that there was no change but this was a satisfactory or neutral experience, and a red X indicates a negative impact or expe-rience.
Demographics and Summary.
Abbreviations: AID, automated insulin delivery; CGM, continuous glucose monitor; DIYAPS, do-it-yourself artificial pancreas system; IOB, insulin on board; TIR, time in range.
A visual summary of thematic analysis for each participant in the AID-IRL study. Abbreviation: AID-IRL, automated insulin delivery in real life.
Discussion
Overall, most participants articulated positive feedback regarding their use of commercial AID systems, particularly around learning curves, increased time in range, and overall quality of life improvements. However, most participants mentioned that their commercial AID experience could be improved with regards to post-meal hyperglycemia. Most participants also indicated their desire for adjustable targets, including more flexibility to have adjustable temporary target settings, and better information regarding insulin on board (IOB). Most AID systems, like traditional insulin pumps, only show bolus-based insulin on board and do not reflect any amount of insulin adjustments made by the system automatically. This may be challenging for users to understand in the future as more commercial systems add in some automated bolus capabilities but still do not display the amounts of correction insulin above or below compared to the neutral pre-programmed basal settings. This concept, known as netIOB, 3 is now normalized in the do-it-yourself artificial pancreas system (DIYAPS) community and may be useful for industry to consider shifting to for greater user understanding of what the system has done over time to adjust for glycemic excursions and predictions of excursions. Similarly, visibility into system predictions may be useful for user understanding of why the system is choosing to adjust insulin delivery the way it is.
Participants also expressed the desire to have the ability to remotely bolus (using a mobile device) and have AID system information available in remote monitoring systems, both for the user themselves and for any caregiver or partner who is permitted to remotely view real-time diabetes data.
Some participants (1, 2 [Control-IQ] and 4 [CamAPS FX]) mentioned the importance of the settings as input values for influencing the outcomes of AID. These users were previous DIYAPS users or mentioned “lurking” in the DIYAPS community, where settings are often discussed and learning from peers is common. 4 The previous DIYAPS users had mixed experience on commercial AID with increased hypoglycemia or decreased time in range (compared to their DIYAPS experience), but both had time in range results that were at similar levels to other participants. Similarly, both mentioned positive experiences overall about the convenience and ease of commercial AID use.
The CamAPS FX user presented an observation that users of a certain type of DIYAPS (Loop 5 ) may benefit the most compared to other DIYAPS users from a transition to a CamAPS FX style AID system where the requirement for settings input and meal management is reduced compared to what they experience in DIYAPS. This user did not think that OpenAPS users with experience doing fully unannounced meals (with or without boluses) 6 would appreciate the same benefits that he predicted for Loop users, due to the differences 7 in the algorithms. However, on the opposite side of AID being able to “learn” user settings, the CamAPS FX user shared an anecdote that the system “learned” that he showers in the morning, causing it to increase insulin prior to that time to compensate for the time off the pump during the shower, which backfired on a morning when he did not shower, causing hypoglycemia.
Systems that “learn” one type of behavior or activity may make it difficult for patients when they change to a different pattern of behavior or activity. There are numerous ways that this could occur—ranging from a change in schedule (changing shower times, weekday vs weekend profiles with regards to school or work or exercise patterns) or a change in diet from high to lower carb or vice versa—and could leave users frustrated with these systems, since the commercial systems often don’t have a way to let users provide input to correct the system’s learning algorithm if it begins overreacting to an observed pattern or otherwise making unsafe dosing decisions.
Most users experience with AID was heavily influenced by their experience with its continuous glucose monitor (CGM). Most had previous CGM experience, and those newer to CGM were more likely to express frustration about alarms and alerts that could be more attributed to the CGM learning curve than the AID itself. Although asked about communication difficulties between CGM and pump, only 670G users expressed frustration with the design of the CGM and its integration or communication in their AID. The transmitter charging design and time, along with the CGM warmup, was mentioned by both 670G users. Overall, these users seemed to experience more sleep disruption and more troubleshooting to keep their system running compared to other AID systems. This feedback seems to mirror other studies 8 findings’ regarding real-world use and continuation of the 670G. For companies that have users switching to AID and using CGM data for the first time, they may need to consider separate training or transition periods for learning to work with CGM data, alerts/alarms, and especially the difference from manual mode to automated insulin delivery mode: users need to understand when they, versus the AID system, should be taking action to adjust for out of range blood glucose levels.
Several users mentioned wanting protection from hypoglycemia even if high corrections are disabled or turned off. This was a large source of frustration for multiple types of system users (Control-IQ and 670G) where users expected the ability to separate hyperglycemia features of the AID from the hypoglycemia protection features of AID—especially for systems like the 670G where users were more often removed from AID mode but leaving them without hypoglycemia protection. This was the one main area where expectations were not met.
Overall, participants were happy with their systems and felt their expectations were largely met, indicating that companies have done a fairly good job in educating and onboarding users with regards to what these systems can and cannot do. However, even with “ideal” outcomes (where real-world time in range and hypoglycemia rates match or surpassed the clinical trial data), commercial AID system users are still doing frequent manual corrections (two to three per day, in addition to mealtime interactions) with hybrid closed loop systems. This is significantly more than should be necessary with automated insulin delivery systems, and this is an area where commercial AID systems should further improve.
Limitations
This commentary covers a small (n = 7) qualitative study that is not meant to be a representative sample of commercial AID system use. The recruitment was based on social media postings (positive and negative) and therefore influenced by those who chose to post publicly on Instagram or Twitter about their experiences with commercial AID. Additionally, participants likely have recall and recency bias regarding their experiences, so these points of frustration may be the most recent, rather than average representations of their experiences on AID. In the future, additional studies assessing new and longer-term use feedback based on adoption and learning curve, both with new and experienced CGM users, would be useful to provide further guidance to industry regarding improvements they could make to facilitate adoption by a larger portion of the diabetes community interested in automated insulin delivery systems.
Conclusion
A small, qualitative study evaluating feedback from real-world users of commercial AID systems demonstrates that most real-world users switching from manual to automated insulin delivery have their expectations met with first-generation commercial AID. Companies should consider integrating real-world user feedback with regards to: improving training materials for new users with less CGM experience, improving flexibility of targets and postprandial algorithm performance in their next-generation system(s), and reducing the number of manual interventions required by users.
Supplemental Material
Appendix_PDF – Supplemental material for Automated Insulin Delivery in Real Life (AID-IRL): Real-World User Perspectives on Commercial AID
Supplemental material, Appendix_PDF for Automated Insulin Delivery in Real Life (AID-IRL): Real-World User Perspectives on Commercial AID by Dana Lewis in Journal of Diabetes Science and Technology
Footnotes
Acknowledgements
Thank you to the participants of the study for their time and feedback, and to Amy Tenderich for conceptualizing the study. The researcher acknowledges she is one of the creators of and contributors to one of the open-source automated insulin delivery systems (OpenAPS).
Abbreviations
AID, automated insulin delivery; APS, artificial pancreas/artificial pancreas system; CGM, continuous glucose monitor; DIYAPS, do-it-yourself artificial pancreas system; SAP, sensor-augmented pump; TIR, time in range.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: DiabetesMine provided funding for the study (researcher’s time and participant honorarium) as part of the Summer 2020 DiabetesMine D-Data ExChange event.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
