Abstract
There are several computer applications (apps) that can be installed on smartphones to assist patients with diabetes in treating their disease. Not only does the range of use of such apps vary greatly, their quality does as well. As part of the DiaDigital initiative of the German Working Group for Diabetes Technology (AGDT), apps are evaluated using a set of criteria and a seal of distinction is then awarded (www.diadigital.de). The information collected in this review process is made public on this website to ensure both the necessary transparency as well as to be able to rapidly adapt voting on further app development. Until now, no such approach on evaluating the quality of diabetes apps has existed in Germany. The hope is that this will also help improve the quality of apps.
Many patients with diabetes want support in coping with their disease during their everyday lives. The extensive availability of smartphones or tablets enables patients to use apps that provide such support. The number of diabetes-related apps is probably in the tens of thousands worldwide. In just one app store (Apple) more than 1000 such apps are available. There is a clear necessity for a way to help control the avalanche of apps. 1 The situation in Germany is better in terms of the number of apps, as only about 100 German-language apps are used frequently. However, for the interested patient this does not yet mean that it is easy to make a selection. There are ways to analyze how many diabetes apps have been downloaded, but none to determine if or how intensively they are used, and therefore how successful they really are.
Apps and their quality vary extensively; a broad spectrum exists in terms of the (programming) effort involved and the maintenance and further development of the app. These range from applications developed by individual patients or programmers who use them to spread their own approaches to apps by corporations which see them as the best way of communicating with their customers. Considering the total financial volume of apps nowadays—and not only medical apps—of more than 1.5 billion euros, app programming has become an industry. New developments in the field of patient treatment (such as automated insulin delivery) via apps will most likely be widely used in the future. The highly complex apps required for such an application, which should always be connected to the cloud, differ considerably from simple apps. In addition, the medical benefit for patients with diabetes varies depending on the app itself although assessing the benefit of the app depends greatly on the personal perspective.
To date, there have been few systematic and structured studies on whether the advantages, such as avoiding acute metabolic derailments and a general improvement in metabolic adjustment, are actually achieved through the use of apps. For example, there have been few publications in diabetological literature to date dealing with apps, but there are current initial critical evaluations, for example, of bolus calculator apps. 2
The high frequency of use of apps shows that they present patients a concrete advantage in everyday life—otherwise patients would not use them. Whether and to what extent the use of apps has a demonstrable medical benefit remains to be clarified. We see a definite need for diabetologists and the diabetes teams to develop competencies when it comes to apps. Most of the inquires received by the German Working Group for Diabetes Technology (AGDT) of the German Diabetes Association come from patients, and the frequency of these inquiries indicates that patients feel that their diabetologists are unable to provide qualified answers to this topic.
The DiaDigital initiative of the AGDT aims to support all patients and groups interested in digitalization and especially in apps. This working group consists of a number of people of different backgrounds (physicians, diabetologists, diabetes nurses, scientists, app developers, and more than 30 diabetes patients who use such apps. We have developed a set of criteria for evaluating apps and providing a seal of approval for those that fulfil these requirements. It must be noted that this set of criteria has been developed by our working group representing a wide range of expertise and backgrounds and is not based on any published previous work.
Current Status of Apps
Apps are a way for patients
to collect, analyze, and present relevant data
to obtain information
to receive warnings
to be reminded of blood glucose measurements or insulin inputs by alarms
to receive support in calculating the right insulin dose with the help of an integrated bolus calculator, etc
to control devices
Good apps support users in an active and self-directed approach to their disease. They help patients adapt their lifestyle (eg, monitoring of nutrition and exercise) and therapy as well as facilitate communication with other affected persons. The rapid development of technical possibilities makes coaching via apps possible nowadays, that is, the patient can speak directly to a diabetes consultant.
Availability of Apps
Any programmer can add an app to an app store without any systematic control of functionality, security, and so on by an independent body. This is the reason for the number and diversity of apps available. However, if apps are used to manage diseases, these can sometimes pose a problem as there is no guarantee that the apps used are of a high quality or that their use does not pose a risk to the patients. If, for example, the algorithm used to calculate the insulin dose contains errors, this can result in life-threatening acute metabolic disorders.
Apps as Medical Products
To avoid these and other risks, all computer programs (including apps) that make statements about the therapy of patients in any form are considered a medical device and must have a CE marking in Europe. However, the process to get this marking takes both time (1-2 years) and money (upward of 500 000 euros), which is why only a few apps have a CE marking.
A general “disconnect” is visible between the regulatory world and the rapid development of technical solutions in the field of smartphones and diabetological technology. If products (here, apps) come to the market after the protracted approval phase, they may already be outdated. This raises the question of how the justified need for safety—paired with the proven benefits of the product—can be combined with the rapid availability of innovative products for patients. It is unclear whether and in what form the legal requirements will change (also at the European level) to take the developments mentioned into account.
Data and Data Security
Overall, data are the core of all apps, and appropriate data handling should benefit the patient. Within the concept of data, all relevant aspects must be considered, in particular data security and compliance with data protection regulations such as the EU General Data Protection Regulation (GDPR). On the one hand, this applies to access to the user’s personal and account information. On the other hand, it applies to associated data and a possible third-party use of this without the user’s consent. It is not absolutely certain that app data remains on the smartphone—it may end up in completely different databases. 3 This is an important aspect when evaluating apps. Since data are considered to be gold today, the handling and interests of app providers must be clearly stipulated.
Apps and the Diabetes Market
In view of the potentially massive global market of hundreds of millions of patients with diabetes, there are a large number of commercial parties interested in the development and distribution of apps. Varying business models exist, including providers—among others—who have apps developed and make them available to patients free of charge to open up a communication channel to patients for interested pharmaceutical companies. It is no coincidence that companies such as Apple and Google are so interested in apps and their use as these global players see an important future market here.
Evaluation of Apps
Apps are widely rated on the Internet but the set of criteria used is often unclear or is viewed critically; this also applies to ratings that have been published in patient journals. Up to now, there has been no official body in Germany that carries out a systematic and independent evaluation of apps. The contribution of technological expertise is vital as well as providing the corresponding infrastructure or platform for the publication of results developed by the project partners (see below).
The question is, who can rate apps and how is it done? Can a diabetes app be rated by a person without diabetes? The very approach of wanting to evaluate apps in some way can, in principle, be seen critically. There are statements that such approaches, which have already existed to evaluate home pages dealing with medical topics, and many apps are probably implementations of home pages at this user level, are a difficult or even hopeless undertaking.
We basically see two different approaches:
The AGDT (ie, the members of the DiaDigital initiative) systematically evaluates apps and publishes them, for example, on its own home page. However, it is neither feasible nor possible to continuously evaluate all apps (and their updates).
Another option is that users (ie, patients with diabetes) evaluate apps transparently using a catalogue of criteria and the results are listed on the home page. Patients, health care professionals, and scientists must agree on an accompanied self-disclosure procedure for evaluating apps. Apps that fulfill such requirements are awarded a DiaDigital seal.
We consider the second approach to be the most sensible and have therefore drawn up a diabetic-specific catalogue of criteria (Table 1). The seal is awarded when the set of criteria (Codex criteria) is fulfilled (Table 2).
Criteria That Should Be Specified for the Respective App to Be Evaluated.
Self-disclosure by the manufacturer for the app. *Codex criteria with product details.
Verification of Self-Disclosure: The DiaDigital Group and the Technology Partner Check the Manufacturer’s Information for Accuracy: The Codex Criteria Considered Essential Must Be Met.
Evaluation by the DiaDigital Group (affected persons, practitioners, and technology partners).
The DiaDigital Seal
It is important that the evaluation of the apps and awarding of the seal follows clear and well-defined rules. In addition, the conflict of interest of all individuals and groups involved in this activity is transparent and the steps (as presented below) are documented.
Manufacturers that have programmed an app for patients with diabetes approach the DiaDigital working group for apps and ask for it to be evaluated. The manufacturers confirm compliance with the criteria by providing information themselves and the DiaDigital working group assesses the accuracy of the information. A technology partner (ztg GmbH; a governmental agency of the state of North Rhine Westphalia; www.ztg-nrw.de) assesses the aspects of IT security in app programming (appendix), such as the (non)encryption of data streams. Users are asked to provide feedback on the functionality and actual benefits. While the individual reviews do not provide complete validation, the considerable number of independent evaluators (at least five diabetes patients und five health care providers [HCPs]) is regarded as a sufficient safeguard to avoid biased views and prevent conflict of interest issues. During regular telephone conferences steered by MK and DD, the evaluation outcome of a given app is discussed among all parties involved in the evaluation process. The seal is awarded only if the criteria are met completely (100%). If the team refuses to award the seal, the manufacturer receives feedback and the opportunity to improve the product. The procedure for awarding the DiaDigital seal is seen as an accompanied, voluntary commitment model. For reasons of practicability and transparency, this process is carried out on an Internet platform where the results of the cooperative evaluation of the apps are also presented. To date, six apps have received a seal after passing the evaluation process successfully and six apps have not.
The manufacturer can declare the seal in the app store and the diabetes association provides the information to the public and their members. The aim is to help potential users to find useful apps and quickly separate the wheat from the chaff.
In the end, such a seal should help the market regulate itself because patients will download and use an app regularly only if it fulfils certain (constantly updated) criteria and is deemed good. This should also lead to an overall improvement in the quality of apps.
We see the following advantages in using such a seal (positive selection):
Not very vulnerable from a legal point of view because only positive statements are made in awarding seals
User perspective is clearly represented
Evaluator competence is developed
Community strengthens itself
Concrete communicable recommendations are made
Intensive reconciliation of interests
Probably delivers the most reliable results
Assessment performed jointly with the manufacturers
Problems With This Approach
Such a seal has its weaknesses. To name a few, a considerable organizational effort is necessary and a functioning structure must be maintained over a longer period of time. This is also associated with costs, covered up to now by the AGDT.
Apps can be seen as highly dynamic—a manufacturer can quickly react to detected defects by means of an update. It is important to consider such updates as part of evaluations. If these refer to older versions, they can quickly become obsolete if an update to the app is performed. For this reason, manufacturers are required to inform us about updates. If an evaluation unearths what seems to be a problem with an app, the manufacturer of this app can regard it as incorrect and take legal action for potentially damaging its business.
Summary
The systematic use of a catalogue of criteria is intended to help to determine the quality of diabetes-related apps. Patients with diabetes and HCPs perform joint evaluations. The information collected about the apps evaluated are presented and stored on the www.diadigital.de home page. Apps that meet the criteria are awarded with a seal.
Footnotes
Appendix
Abbreviations
AGDT, German Working Group for Diabetes Technology; GDPR, General Data Protection Regulation; GTC, general terms and conditions; HCP, health care provider.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
