Abstract
Background:
The Nightscout Project is a leading example of patient-designed, do-it-yourself (DIY), open-source technology innovations to support type 1 diabetes management. We are unaware of studies that have described the evolution of patient-driven innovations from the Nightscout Project to date.
Methods:
We identified patient-driven, DIY innovations from posts and comments in the “CGM in the Cloud” private Facebook group as well as data from Twitter, GitHub, and the Nightscout website. For each innovation, we described its intent or its unaddressed need as well as the associated features and improvements. We conducted a thematic analysis to identify overarching patterns among the innovations, features, and improvements, and compared the timeline of innovations in the DIY space with the timing of similar innovations in the commercial space.
Results:
We identified and categorized innovations in Nightscout with the most commonly appearing themes of: visualization improvements, equipment improvements, and user experience improvements. Other emerging themes included: Care Portal support, safety, remote monitoring, decision support, international support, artificial pancreas, pushover notifications, and open-source collaboration.
Conclusions:
This rapid development of patient-designed DIY innovations driven by unmet needs in the type 1 diabetes community reflects a revolutionary, bottom–up approach to medical innovation. Nightscout users accessed features earlier than if they had waited for commercial products, and they also personalized their tools and devices, empowering them to become the experts of their own care.
Introduction
The Nightscout Project has gained notoriety in the broader diabetes and health-care communities as an example of patient-driven innovation in personalized medical technologies. It started in 2013 when the father of a four-year-old child with newly diagnosed type 1 diabetes (T1D) recognized a need to monitor his child’s blood glucose (BG) levels in real time while away at school. 1 He developed computer code that would send his child’s BG readings from a Food and Drug Administration (FDA)-approved continuous glucose monitoring (CGM) system to the computing cloud so that he could see the glucose data remotely on a laptop, mobile phone, or smartwatch. He later shared this code with other patients and caregivers in the T1D community.
This individual act of sharing has led to a broader movement of open-source collaboration among a growing community of patients with T1D and caregivers. Since then, the community has collectively created or contributed to a number of do-it-yourself (DIY) technical innovations to support T1D management. This collaboration has been documented through the Nightscout Project website, 2 code repositories on GitHub, 3 and a private Facebook group called “CGM in the Cloud” 4 where community members ask for help and troubleshooting as well as share their contributions. As of July 2019, the Facebook group had over 30 000 members, with more than 4000 members outside of the United States. However, it is important to note that these examples illustrate a limited subset and are not the only instances of collaborative activity among patients with T1D and caregivers.
The DIY movement started with CGM remote monitoring solutions and then evolved into additional software and hardware systems and artificial pancreas systems (APS) such as OpenAPS, Loop, and AndroidAPS. An exploration into the changing landscape of DIY technology to manage T1D is necessary to (a) highlight a growing maker movement of patient-driven design, (b) educate healthcare providers, patients, and caregivers on the types of DIY technologies available, and (c) inform the medical/healthcare community of the results of an open-source, bottom–up approach to healthcare innovation. Previous publications describe the real-world use of Nightscout and related technology,1,5-13 but we are unaware of publications that have described and thematically analyzed the evolution of patient-driven innovations from the Nightscout Project. Therefore, the objectives of this study were to identify and describe the different types of patient-designed DIY innovations in the T1D community focused on Nightscout, and to describe the timeline of innovations in comparison with similar features available in commercial products. In addition, we evaluated metrics related to code use and contributions to open-source code repositories for Nightscout.
Methods
To identify patient-driven DIY innovations in the T1D community, we reviewed posts and comments in the “CGM in the Cloud” private Facebook group, tweets on Twitter, information from the Nightscout Project website, and documentation on GitHub, a web-based repository that allows developers to store, manage, and document changes to their code. This research project was approved by the University of Michigan Institutional Review Board with a waiver of informed consent authorized for the project. The principal investigator (JML) was granted administrative privileges for the private Facebook group, which was acknowledged in the IRB application. No individuals were actively recruited as this was a secondary analysis.
Primary source data were Facebook posts and comments (both text and visual content), extracted using the Facebook application programming interface (API) available for administrators of Facebook groups. An access token was obtained with developer privileges, and an automated data pull was devised in R programming language by the project team encompassing May 2014 to February 2018. This end date was due to changes in the Facebook API. Data from the Facebook group beyond February 2018 as well as additional data from Twitter, Nightscout Project website, and GitHub were manually searched and extracted by the study team ad hoc during the coding process.
We focused our coding process on major innovations, defined as new software releases or changes that addressed new goals and/or achieved new functionalities. We also detailed specific features and improvements associated with the innovation where applicable. We defined major changes as updates or additions that dramatically affected functionality or standalone updates to the code. We excluded any features in releases that were bug fixes or general software updates.
For each of the innovations identified, we described the intent or unaddressed need that prompted the innovation. We then conducted a thematic analysis in which we characterized overarching patterns among the innovations, features, and improvements by assigning one or more codes. Collectively, the codes served as meaningful groupings that allowed us to see similarities and differences across the innovations and features. 14 We assembled the codes into a shared codebook, which served as a tool that defined and provided examples of each code, as well as any exclusions, for analysis consistency. The codes and emerging themes were mutually compiled and agreed upon by four research team members. Any disagreements in regard to the assignment or definition of a code, and whether new codes should be added or revised, were discussed thoroughly until there was a consensus. Once the codebook was applied multiple times with no new codes emerging, the codes were considered a valid representation of the data. 15 We also compared the timeline of innovations in the DIY space with the timing of similar innovations in the commercial space by identifying features and devices that were FDA-approved.
Finally, using the GitHub website, we evaluated metrics related to computer code use and contributions for Nightscout. GitHub hosts computer code for each system that is open source, which means that any individual accessing the website can review, modify, or enhance the code. Metrics include contributors, which are individuals who make changes to code that subsequently become part of the software project; 16 commits, which are revisions to the code 17 ; and releases, which represent packages of code that can be downloaded. 18
Results
The table in the appendix provides a list of identified innovations, the need or purpose, and a description of the features/improvements, presented in chronological order. Table 1 lists the frequencies of each code and how each code is defined. The innovations fell into 11 thematic categories with many innovations classified under multiple themes. We describe the themes in descending order of frequency.
Code Definitions and Frequencies.
Visualization Improvements
Visualization improvements included any changes that helped users to better visualize, understand, and interpret their data. Initially, this included the ability for users to access additional data not traditionally available, such as raw BG levels during sensor warm-up and calibration, predicted BG levels, and self-management information such as insulin-on-board (IOB), carbohydrates-on-board (COB), current basal rate, and exercise events. There were also design modifications that allowed users to more easily make sense of their data and trends at a glance, such as the addition of HIGH/LOW labels, the use of colors to indicate low, in-range, and high BG levels, or the juxtaposition of the most recent glucose reading with a graph of glucose over time on a Pebble watch. Another added feature was the ability for individuals to examine the functioning of the system or its components (ie, whether data have stopped updating or uploader battery is low). One particularly novel innovation was the creation of a Pebble watch face that allowed users to simultaneously view glucose data for two different individuals with diabetes.
Equipment Improvements
Equipment improvements included enhancements to hardware and/or associated software. Hardware improvements included general improvements such as increased portability of the Nightscout “rig” as well as use of more cost-effective tools (eg, non-contract Android phone), while software improvements included simplified onboarding processes and increases in speed or database performance. Such improvements also allowed users to record, access, and integrate data across interfaces (eg, Care Portal for documenting treatments; mobile and smartwatch applications for viewing data) as well as incorporate data from multiple brands of CGM and insulin pumps into a user’s Nightscout website.
User Experience Improvements
User experience improvements included any changes that enhanced the user’s interaction with the technology. This includes opportunities for personalization like the ability to add a custom title for the Nightscout website, a custom treatment profile, or custom alerts and alarms for user-defined glucose thresholds. This also included improvements in accessibility, such as a “colorblindfriendly” theme that increased accessibility for colorblind users, or integration with Amazon Alexa, which would recite the BG values, increasing accessibility for visually impaired users.
Care Portal Support
Care Portal is an add-on that provides web-based logging and tracking of diabetes care on the Nightscout website to enhance CGM data interpretation. 19 Care Portal support features included the ability to log additional contextual data such as grams of carbohydrates eaten, insulin doses given, temporary basal rates, exercise events, and infusion site and sensor placement. Data entered into Care Portal could then be viewed on the Nightscout display along with CGM data and used to provide further insights for users such as insulin dosing decision support based on the relationship between current BG, IOB, and COB.
Safety
Safety included any features that placed safeguards against potential equipment malfunctions or provided users with preventative measures to avoid potentially dangerous situations. These included features like the addition of error codes that would display when the CGM encountered an issue, and “stale data” indicators that notified users when the data stopped updating to the Nightscout website. Users could also set their own thresholds for what constituted “low” and “high” BG levels, and set alarms for either current or predicted low BG levels. Users also had the ability to determine who was authorized to update and/or view their data on the Nightscout website.
Remote Monitoring
Remote monitoring was the ability to monitor glucose levels in real time from afar. Remote monitoring innovations began with having the ability to access and monitor a child’s real-time glucose levels while the child was at school via a computer, and later, a mobile phone. It then evolved into developing Dexcom Connect, an Android app that received data from a CGM transmitter and pushed the data to the computing cloud continuously, so that users could easily monitor BG levels. The Nightscout Pebble Watch app further offered users the ability to monitor BG levels on a smartwatch. Later, remote monitoring capabilities expanded, allowing users to view their Dexcom Share or Medtronic Enlite CGM data on their Nightscout website.
Decision Support
Decision support was used to describe features or innovations that provided additional information to guide users’ treatment decisions. This included specific treatment recommendations as well as data or any information that could be used for decision-making. For example, an early decision support feature was the addition of the “confidence interval cone,” which displayed the predicted range of BG (based on a 90% confidence interval) on a graph that helped users to make more informed decisions based on an understanding of the range of possible values versus a single predictor line. Later, an optional, experimental IOB-COB website was developed that used an algorithm to predict BG levels over the next three hours based on Care Portal data and a user-defined therapy, and allowed users to check if their BG was changing as expected based on current insulin and carb activity. 20 Also, using information from Care Portal entries, Bolus Wizard Preview (BWP) offered treatment recommendations to users by suggesting temporary basal rates and carbohydrate corrections and alerting users to consider testing and correcting when BG levels were moving out of range. Additionally, the BWP innovation went beyond merely suggestions and took it one step further by actively snoozing high-blood-glucose alarms when there was adequate IOB. A number of reports were also developed that display day-to-day summary data from Nightscout providing higher level insights to users.
International Support
International support included any features that provided support or assistance for users outside of the United States. International users initially experienced distinct barriers in their user experience that prevented them from fully taking advantage of the DIY technology. As a result, the Nightscout team added features to allow users in other countries to better access and use this technology, including the addition of translations into more than 20 non-English languages. They also offered options to display the BG levels in millimoles and to display 24-hour time, so that international users could best interpret their data and interact with the technology.
Artificial Pancreas
Artificial pancreas included any innovations supporting the functionalities for DIY automated insulin delivery systems. DIY closed-loop systems enable insulin pumps to automatically adjust basal insulin delivery in response to real-time glucose data from the CGM. OpenAPS was the first of the systems to be developed and released, followed by Loop/LoopKit, and then finally, AndroidAPS. For example, the Nightscout “Grilled Cheese” release allowed users the ability to monitor OpenAPS and Loop in Nightscout. The “Ice Cream” release added a feature to better visualize Loop data.
Pushover Notifications
Pushover notifications included any innovations that enabled the ability to receive notifications across devices. For example, the “Brownie” Nightscout release offered integration with a service that pushed messages to a mobile phone, Pebble Watch, and/or desktop when there was an update to Care Portal. Later releases offered additional features such as the option to get Pushover notifications when glucometer BG values were entered as calibrations, and the ability to make alarm/notification groups to streamline the number of alerts.
Open-Source Collaboration
Open-source computer code is publicly accessible, allowing individuals to share and modify the code. The Nightscout code was initially shared among community members and later made open source via GitHub so that others could benefit from the technology. Once it was released, other interested users took it upon themselves to make changes to the code. Because of this online collaboration, the technology became the product of not just one user, but many users who dedicated their time and energy toward testing the code, making improvements, and creating innovations.
In addition to our thematic analysis of features and innovations, we also explored the timeline of developments for select key innovations, comparing DIY features with similar features from commercial products (Figure 1). One feature was first released in a commercial product, while the remaining four features were first available through DIY solutions. Among the features first available via DIY technology, the median time between a DIY feature’s availability and a similar commercial release was 17 months, with a range of 0.5-21 months.
Comparison of launch of DIY and commercial tools and features.
Finally, metrics from GitHub represent community involvement in developing and utilizing the Nightscout code with 132 contributors and over 5000 commits.
Discussion
To our knowledge, this is one of the first studies to describe the evolution of Nightscout DIY technology innovations and to identify user needs or gaps that led to their development. Because this was an analysis of the Nightscout system, for which the major purpose is to view data remotely, the majority of documented innovations were related to data visualization. Data visualizations provide users with greater insights from the CGM through multiple views, additional contextual data, and customized design. However, over time, the system evolved to include more advanced features, such as integration of multiple diabetes devices and interoperability across desktop/mobile platforms, as well as greater support for the user in terms of notifications, clinical decision support tools, and integrated viewing of OpenAPS, Loop, and AndroidAPS.
Based on the nature of innovations, it is clear that autonomy and choice were priorities of the Nightscout community. The Nightscout Project supported data collection from multiple diabetes devices that could be viewed across mobile and wearable platforms (Android, iOS, Pebble Watch). Nightscout overcame the limitations of the commercial systems that left data siloed inside each device ecosystem. Users clearly wanted interoperable systems so that they could access all of their personal data together. Furthermore, the developers and designers of Nightscout had the freedom to design a multitude of mobile applications and watch faces that would serve unique user needs. For example, the DIY system provided the opportunity for users to simultaneously monitor glucose trends for two children with diabetes on a single Pebble watch face, to enact personalized voice controls, and to adapt the code for all Android phones, including those that did not require costly network contracts, thus increasing flexibility and affordability.
Another area of need fulfilled by Nightscout was the opportunity for continuous development and enhancement of tools, facilitated by the fact that it was designed as an open-source project. Because it was open source, anyone from across the globe could download the code and contribute, accelerating the speed at which the innovations matured and developed. This no doubt contributed to the wider reach of the technology over a relatively short period of time. The placement of code on GitHub was instrumental in its widespread sharing, and the GitHub metrics demonstrate the use and reuse of the code over time.
It is notable that a significant priority area addressed by Nightscout was the issue of safety. A number of innovations included features to help users assess whether the system was functioning properly. Such safety features include alarms for “stale data” to warn the user if data have not been updated recently, display of CGM error codes, and modifications to the color schemes of the data visualizations to highlight the most important pieces of information.
Our comparison of the timing of key features between Nightscout and the corresponding commercial solutions fits the theory of innovation called lead user innovation.21,22 Lead users are individuals who respond to needs and gaps in available tools and services by developing the solutions for themselves. Eric von Hippel, Professor of Technological Management in the MIT Sloan School of Management, has studied and developed the lead user innovation framework across a range of industries. Products developed using this framework are found to be more innovative and generate more revenue compared with products developed using traditional methods of innovation. As a result, a number of large companies have applied this framework to support their innovation.23,24 Examples of successful lead user innovations in the commercial domain include mountain bikes, kayaks, and even Weight Watchers, which was founded by a woman who initially organized a group of friends to offer support and accountability in her weight loss journey and went on to become a top-rated commercial weight loss program. 25 We cannot prove that the DIY movement spurred the commercial development of T1D tools, but clearly, there were needs and gaps not addressed by commercially available technology, and the DIY movement facilitated access to useful features and tools, often well ahead of the commercial timeline.
There are limitations to our study. First, the majority of our data came from the “CGM in the Cloud” Facebook group. However, we note that the Nightscout Project created this group to share their contributions to the community. Secondly, the unit of analysis for the code frequency table was at the level of major innovations and did not account for the frequency of posts in the Facebook group discussing innovations. There may also be differences of opinion in what could be considered a major innovation. Furthermore, the timeline is based on when innovations were shared on the Facebook group, Twitter, or on the web, but could have been created before the time of the posting. Additional innovations may be missing because they have not been shared widely online.
Conclusion
The findings from this research can be applied to understand other innovations in medical technologies. The development of patient-designed DIY innovations driven by unmet needs in the T1D community reflects a novel, bottom–up approach to medical innovation. Nightscout users not only accessed features earlier than if they had waited for commercial products to reach the market, but they also personalized their tools and devices for T1D management. In turn, Nightscout empowered them to become the experts of their own care.
Supplemental Material
NG_Appendix_-_REVISION_MN_1 – Supplemental material for Evolution of Do-It-Yourself Remote Monitoring Technology for Type 1 Diabetes
Supplemental material, NG_Appendix_-_REVISION_MN_1 for Evolution of Do-It-Yourself Remote Monitoring Technology for Type 1 Diabetes by Michelle Ng, Emily Borst, Ashley Garrity, Emily Hirschfeld and Joyce Lee in Journal of Diabetes Science and Technology
Footnotes
Acknowledgements
The authors would like to thank the Nightscout Foundation for their support of this project.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: JML is a consultant to T1D Exchange (formerly Unitio, Inc.) and receives grant funding from Lenovo. She has no other relationships or activities that could appear to have influenced the submitted work. The other authors have no conflicts of interest to report.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded through a Patient-Centered Outcomes Research Institute (PCORI) Eugene Washington PCORI Engagement Award (1442-UMich) and the C.S. Mott Children’s Hospital Charles Woodson Clinical Research Initiative.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
