Abstract
Objective
In this observational study, we determined the distribution of mHealth applications (apps) in Germany using data from the FeverApp registry.
Methods
The registry data were processed to assess general monthly trends in app distribution, and a seasonal autoregressive integrated moving average model was decomposed to investigate time series. A sample comparison was made matching data from cold-called against self-registered distributers of the FeverApp.
Results
Among 881 pediatric and adolescent medical practices, 27,300 app users were recruited between 2019 and August 2023. The number of monthly recruited users increased steadily. A seasonal trend was observed, showing a higher distribution in winter months. Self-registered pediatric practices did not recruit significantly more app users than cold-called practices, with approximately every 25th family recruited in both groups.
Conclusions
The trend of more app sign-ups during winter is likely related to the flu season in Germany. Intrinsic and extrinsic motivational factors of the practices seem to have a large impact on the distribution. We observed a positive trend in the app distribution. Seasonal febrile infections and individual distribution methods among practices influence the distribution of the FeverApp in Germany. Family factors may have a greater influence than the motivation of distributing practices.
Introduction
Mobile health applications (mHealth apps) are defined as “medical and public health practices relying on mobile devices, such as cell phones, patient monitoring systems, personal digital assistants and other wireless devices.” 1 In a European comparison, Germany lags behind in the use of digital health services. 2 With entry of the “Digital Health Care Act” in Germany on December 19, 2019, the “app on prescription” for patients was introduced into the health care system. This means that approximately 73 million individuals insured in the statutory health insurance system are entitled to a supply of digital health applications. A complex field of tension arises with the diverse possibilities of technology and the different interests of many stakeholders. However, the enormous benefits of mobile technologies when they are used appropriately are also clear. 3 Digital technologies such as telemedicine services, video consultations, teleconsultations, and digital health applications, as defined by law in the German Social Security Code V, are now part of standard health care. These technologies can be used by medical and nursing staff as well as by patients.
Although barriers and facilitators regarding the implementation of such services are being researched, reports regarding the requirements that must be met for optimal integration of mHealth technologies into existing care processes in everyday practice are only available in isolated cases. 4 One of these reports involves the FeverApp. 5
The FeverApp is an mHealth app that allows parents to record, track, and manage children's fever events and symptoms. By providing scientific information based on current guidelines of the leading pediatric societies in Germany (The Professional Association of Pediatricians and Adolescent Doctors [BVKJ e.V.] and the German Society for Pediatrics and Adolescent Medicine [DGKJ e.V.]), the FeverApp helps parents to better understand and safely manage fever in their child. The goal of the FeverApp is to create a model registry through families' self-documentation of fever management so as to draw conclusions about guideline implementation. The app is aimed at educating parents about core guideline information, including that fever is not a disease but a sign of the immune system fighting the underlying causes.6,7 In addition, parents are informed that the management of fever depends on the age and symptoms of the child and are encouraged to document the course of the child’s fever through the FeverApp. If there are warning signs, the app prompts the recommendation to visit a doctor. If the course is not dangerous and there are no warning signs, unnecessary visits and strain on the resources of medical practices can be avoided. It is not always necessary to immediately see a doctor or to take medication with a feverish child. The decision to see a doctor is not promoted by the app but is made by the parents. The FeverApp provides information that is unchanged from the guidelines, which is not intended for the diagnosis, treatment, or causes of fever but is merely for the documentation of fever progression and the provision of information; therefore, information provided by the app does not fall under the Medical Device Regulation of the European Parliament. The authors actively decided not to classify the FeverApp as a medical device because this would imply a risk factor if taking part in this registry study and would require enforcement of development restrictions for a research tool.
There are two factors to consider when distributing a product: the logistics of actually delivering the product to the customer, and the distribution channels, which describes a system of intermediaries or multipliers. 8 Because the FeverApp is a mobile application, the logistics behind its distribution are negligible. Mobile applications like the FeverApp can be distributed in an automated way via application marketplaces, mainly, the Google Play Store for Android and the Apple Store for iOS devices. Accordingly, the focus below is on the distribution channels.
The distribution channels of mHealth applications can vary widely because they can be considered a service as well as an end product. Apart from health insurers, physicians and hospitals are also considered to be efficient distributors of mHealth applications; pharmacies and health care websites are no longer considered to be effective distributors of such applications. 9
In addition to the identification of gatekeepers, equal and joint decision-making about an appropriate approach in the context of mHealth application implementation can be beneficial. 10 Our goal in observing the distribution of an mHealth app, using the example of the FeverApp, was to provide insight regarding possible internal and external influencing factors as well as an overall description of app distribution on a large scale.
Methods
FeverApp registry
The FeverApp has been available to download since the end of August 2019 but was access code-protected for scientific purposes. As of August 2023, the access code has become optional; hence, the FeverApp is freely accessible to everyone. In this observational study, the registry data include FeverApp users from pediatric and adolescent medical practices since the app's inception (in 2019) through August 2023.
Dissemination
According to the registry protocol, the distribution channel of the FeverApp was initially intended to comprise selected pediatric practices that participated in a data validation study. 5 These reference practices should deliver the same data as data collected in the app-based registry to ensure physician-validated data among the data collected from parents in this second phase. Despite this aim, the unstructured and invalid manner of fever data collection in the daily routine of pediatric practices has generated the need to create an additional practice registry, which was finally implemented in four practices using the same ambulatory information system. Owing to the huge effort involved for practice personnel in documenting each existing fever event for each attending family, in addition to technical challenges, inclusion of additional practices was not feasible at the initial stage. 11 Because several different ambulatory physician information systems are in use, this often does not easily allow for additional questions or exportation of the collected data. Therefore, it was necessary to develop separate macros for each system as well as separate applications, including a Microsoft Access database (Microsoft Corporation, Redmond, WA, USA), although a separate application further increases the workload even for highly motivated practices. It should be stated that the current information systems in German pediatric practices is not very accessible for extensive research questions owing to compatibility issues.
To start the recruitment of families, invitations in pediatric practices via a family-specific code were omitted in July 2020 and replaced with a practice-specific code, which kept the information of each family anonymous. Without the need to participate in an additional validation study and with the only demand being to share the practice access code with children’s parents, many more pediatric practices could participate. Because no practice data were used, there was no need for a separate paper-based informed consent form for participating families. Parents agreed electronically to make their data available in the FeverApp registry, which considerably decreased the workload for pediatric practices.
After this major change, the number of participating practices increased drastically. From mid-2020, all users were tied to practice codes, which ensured anonymity because there is no link to medical records. There were no specific eligibility criteria for pediatric practices or families regarding the distribution of the fever app because this was a registry-based study aiming to represent as much of the population as possible.
As part of the FeverApp process, families are seen by the supervising pediatrician in the long term and, if necessary, in the context of intervening consultations for dealing with fever. Low-threshold intervention components (e.g., information brochure and repeated pediatric consultations during the examinations) were deliberately combined with components that require active use by the parents (e.g., information on the website and in the app). The app was usually distributed during practice visits but practices were free to use individual methods, and there was no specific training or standardization of the distribution.
The FeverApp and its website have been translated into the native languages of the largest immigrant groups in Germany. Through this bundle of measures, the FeverApp has the potential to reach all families nationwide and thereby reduce health disparities to bring about a reduction in antibiotic prescription use and health resource utilization.
Specifically worth mentioning is the COVID-19 pandemic and lockdown in Germany during March 2020, which brought about delays and caused interference in recruitment owing to declining numbers of infections and visits to pediatric practices, as well as reorganization of face-to-face events, among other things. The original intended validation methodology via StuRP (Students in Reference Practices) could not be carried out because students could not visit practices. As of July 2020, data validation via practice registries was supplemented with other validation techniques. Contrary to the original plan, priority recruitment through pediatricians was maintained to increase scientific controllability and transparency. The registry has established a solid foundation for the further evaluation of data.
No patient details are included in the data for this research, with all having been de-identified in the research process; therefore, informed patient consent was not required. All research was carried out with the approval of the ethics commission of Witten/Herdecke University in 2018 (139/2018). The study was conducted in accordance with the Helsinki Declaration of 1975, as revised in 2013. The reporting of this study conforms to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. 12
Data source
Technical data from the FeverApp Registry (DRKS ID: DRKS00016591) were used for the analysis, which was conducted using ecological momentary assessment of childhood fever episodes via the FeverApp, implemented by users as a documentation and information tool. For a certain period, the app was only available through a cooperating pediatric practice, with interested parties acquiring access to the FeverApp via a dedicated practice code. Only data from this period were used for the analysis.
The inputs and interactions among the different pages of the app are stored locally in the app in an open source JavaScript database, PouchDB, which is synchronized with an Apache CouchDB in online mode. The latter database is centrally located on the server of Witten/Herdecke University and is converted to a Mongo DB on a daily basis. Multiple relational data tables (CSV format) are extracted as needed using structured query language scripts and processed in IBM SPSS version 26 (IBM Corp., Armonk, NY, USA) or RStudio Build 764 (The R Project for Statistical Computing, Vienna, Austria). These data represent the FeverApp registry. 13 For testing purposes, only special testing access codes have been used to ensure that only true observational data are collected in the routine procedure.
All regions of Germany are represented. The FeverApp was both advertised via the BVKJ and distributed in Baden-Württemberg through the Association of Statutory Health Insurance Physicians. When compared with the overall figures for Germany, the state of Baden-Württemberg was considered separately. Here, 60% of all pediatric and adolescent practices were asked to use the app in a randomized controlled trial, via a letter from the Association of Statutory Health Insurance Physicians, and were sent a “starter package” with 150 flyers each. In the states of Saxony-Anhalt, Saarland, and Baden-Württemberg, a proportion of all practices in the state that were not already cooperating with the FeverApp project at that time were included in a sample comparison.
Data analysis
To record the app distribution process in this longitudinal analysis, the data of FeverApp users were exported from the registry. We observed when a user profile was created and with which doctor's practice it corresponded. The practice code of the pediatric practice can be extracted from the respective family code, which is linked to each user profile. The variable, creation of a user profile (role_created), was used as the measurement time. Together with the practice code, a recruitment history can be generated at a monthly level, which shows how many user profiles were created in which month by how many practices. Missing values were not considered in this observation. From the difference, an indicator could be generated that showed how many users per practice were recruited on average, in relative terms: the distribution quotient = number of recruits/number of active practices per month.
The distribution quotient is not only useful for assessing the absolute recruitment numbers, which increases with continuous recruitment of new, cold-called practices, this quotient also permits quantification of the overall recruitment efficiency of the practices. The higher this value, the more effectively the practices recruit.
Longitudinal analysis was executed using a SARIMA (seasonal autoregressive integrated moving average) model. The SARIMA model can be used to investigate time series and non-stationary data with recurring intervals. 14 A SARIMA model can be decomposed using seasonal decomposition. This allows for breaking up time-series data into seasonal, trend, and residual components. By analyzing these components, further information can be obtained. The SARIMA model was generated and decomposed using the forecast package in RStudio. 15
When considering cold-called medical practices in the states of Saarland, Saxony-Anhalt, and Baden-Württemberg, the data were first cleaned with regard to self-registered practices. Then, active, cold-called practices were identified and compared with the absolute number of practices contacted. An active practice was defined as any practice having more than one active user role; it can be assumed that the first user role was created by the practice owner for testing purposes.
In the special case of Baden-Württemberg, the self-registered practices were also compared with new, cold-called ones by the Association of Statutory Health Insurance Physicians. In addition to the absolute number of flyers received, the absolute recruitment numbers were calculated according to the number of flyers received by each practice to determine how many distributed flyers led to successful user registration and thereby obtain an indicator of recruitment efficiency. The difference in recruitment between cold-called practices and self-registered practices was also examined inferentially for significant differences using a Mann–Whitney U test. A non-parametric test was used because the data in these populations had skewed distributions.
Results
General distribution
The FeverApp was distributed to 27,300 users through 881 pediatric and adolescent medical practices. The mean age of FeverApp users was 35.69 years, with most (78.29%) users stating that they had the role of the mother and considerably fewer users stating that they had the role of the father (18.59%) or other (3.12%).
The number of monthly recruited FeverApp users and actively recruiting practices increased steadily, with a clear increase during the fourth quarter (Q4) 2020 (Figure 1). Here the impact of the FeverApp mid-2020 recruitment strategy can clearly be observed, in which the FeverApp was opened to more practices, with a delay of approximately 1 to 2 months. Therefore, the point of data analysis was chosen from there on because the low number of cases before that point would lead to immense variance in the data.
Distribution history of the FeverApp.
Between November of 2020 and March 2021, an increase in created user profiles was seen, which then leveled off again in the summer months. Starting from winter 2021, a trend was observed in that there were more user registrations in the winter months than in the summer, peaking at 1594 registered users from 291 practices in December 2022. The average number of monthly recruited users from Q4 2020 onward was 647.81 (standard deviation [SD] = 334.15) from an average 180.14 (SD = 92.66) practices.
The number of recruiting practices increased permanently, but the distribution quotient seemed to differ over time, i.e., more practices recruited users overall, but individual practices recruited different numbers of users, on average. The mean distribution quotient observed from Q4 2020 onward was 3.77 (SD = 0.89).
Additive time-series model
Decomposition of the SARIMA model can give greater insight into the distribution of the FeverApp (Figure 2). The observed part of the model represents the data as a whole and does not give further information compared with the general distribution that has already been described. Auto regression accounts for any trends in the data over a long period by regressing future values on past values. This revealed a steadily increasing trend, with a small spike in mid-2022.
Decomposed SARIMA (Seasonal Autoregressive Integrated Moving Average) model of the distribution history of the FeverApp.
The seasonal component clearly reinforced the observations regarding the distribution history of a highly increased number of app users in the winter months of each year. Hence, there is definite seasonality in the data, causing the distribution to fluctuate from −100 new app users in the summer months to more than 300 in the winter months, forming a recurring pattern.
A random factor in the data could clearly be observed in the last part of the decomposition. The variance that cannot be explained by the aforementioned parts of the time-series model showed a downward trend at the end of 2020 and 2021 and an upward trend at the end of 2022.
Cold calls
Of the 341 practices contacted via cold calling, 100 (29.3%) actively participated in the FeverApp project, with most practices located in Baden-Württemberg. In all three federal states, a response rate of 27.3% to 31.0% was observed, which did not differ significantly (Table 1). The average number of recruited users differed among federal states. In particular, Saarland stands out, with a mean value of 25.2, resulting from two particularly active practices in this federal state, with 64 and 72 recruits each.
Distribution of the FeverApp in three federal states with cold-called practices.
Distribution of the app in Baden-Württemberg
In the following, the difference between self-registered practices versus cold-called practices was examined in terms of recruitment efficiency (Table 2). Using Baden-Württemberg as an example, existing practices had a clear lead in time and a considerably higher absolute number (mean =51.3, SD = 51.9) of recruited families than new, cold-called practices (mean = 8.2, SD = 6.5); therefore, only the percentage recruitment efficiency was compared.
Comparison of distribution efficiency (percentiles) of the FeverApp between existing, self-registered and new, cold-called practices in Baden-Württemberg (BW).
There was a mean difference in the recruitment efficiency between self-registered (n = 35) and cold-called practices (n = 75), with 9% of flyers leading to successful recruitment for the former, compared with 6% for the cold-called practices, which was not significant (95% confidence interval). The median was 4% for both groups, i.e., each 25th invited family was recruited. Cold-called practices had a spread from 1% to 26% and self-registered practices had a spread between 1% and 65%.
Discussion
Most FeverApp users (78.29%) are the mothers of children. Considering that the app is distributed through pediatric practices, in Germany approximately 92% of children between age 0 and 3 years who make pediatric visits are accompanied by their mother. 16 These numbers are in line with related research. Nevertheless, we observed that in some families, both mother and father used the FeverApp, which had an impact on the percentage distribution in comparison to pediatric visits alone.
The increase in the absolute number of app installations rose sharply at the end of 2020. The effects of the FeverApp distribution strategy can clearly be seen from the middle of 2020, which occurred with a delay of approximately 2 months. This 2-month delay can be because the practice first must learn about the FeverApp after being invited and before it can be forwarded to parents. The parents then have to install the app and create a user account before they appear in the data.
Overall, we observed a trend of more sign-ups during the winter months than in the summer. This is likely related to the fact that the winter months are the flu season in Germany and febrile infections are more prominent in the minds of both physicians and parents. Therefore, the season is one factor that influences the number of app installations. 17 Parents who have previously received an access code for the FeverApp but have not yet installed it could also be motivated to install the app if their child develops a febrile infection. However, the registry can only include the data of families after installation of the app.
The large spike at the end of 2022 in the recruitment quotient (Figure 1), as well randomness (Figure 2), could be attributed to the exceptionally strong spread of febrile infections owing to rebound effects from the earlier COVID-19 lockdowns in Germany. 18
Furthermore, a trend was observed in that with an increased number of practices, the distribution quotient decreased, which can be seen as another factor in the distribution efficiency. This may be owing to several reasons. Practices initially engaged in much more personal and face-to-face interaction with patients from the area surrounding Witten/Herdecke University. Furthermore, with expansion of the project and simultaneous inclusion of all federal states, many more practices were cold called, which resulted in a good overall participation rate of approximately 30%, if one takes the results from Baden-Württemberg, Saarland, and Saxony-Anhalt as an indicator for the other federal states. However, the rates of recruitment efficiency also showed that some individual practices were very efficient; these were outliers and recruited many times more users compared with the average, up to 65% compared with a median of 4% (Table 2). Thus, high-performing practices that seem highly motivated could be another important factor and should be further investigated in the future. Of course, it would be ideal if all practices signed up for the FeverApp project under their own initiative, but this is not realistic. Nevertheless, the absolute data still increased with effective cold calling, which can lead to more robust registry data and improved research findings.
The wide range of distribution quotients shows that there are many unseen factors at play when distributing an mHealth application in Germany, which should be evaluated further. In particular, the intrinsic and extrinsic motivational factors of practices seem to have a great impact on the distribution. Among other things, this can be seen in a comparison of the three federal states, amongst which Saarland showed a stronger recruitment efficiency than the other two states. On the one hand, this can be attributed to bias owing to different case numbers. On the other hand, extremely motivated practices can be observed. It is difficult to make out exact measurable reasons for this; however, a quality circle was conducted in Saarland shortly before the data collection, which could have had an influence on the intrinsic recruitment motivation of practices. It seems useful to point out the scientific basis of the FeverApp project when recruiting.
Conclusion
A positive trend in the distribution of the FeverApp in Germany was observable. Seasonal febrile infections and individual distribution methods among clinical practices influence the distribution of the FeverApp. Family factors may have greater influence than the motivation of distributing practices.
A next research step could be to examine the distribution strategies of individual physician practices to determine which are most effective, as well as the retention rate of FeverApp users and their experience with different recruitment strategies.
Footnotes
Acknowledgements
The authors would like to thank all participating practices and FeverApp users for their continued support of this research.
Author contributions
Moritz Gwiasda contributed to the study conception, design, and execution as also to the data acquisition, analysis, and interpretation. Larissa Rathjens, David D. Martin, Ricarda Möhler, Silke Schwarz, and Ekkehart Jenetzky participated in the study conceptualization and draft review, revision, and critical review of the article.
Data availability statement
The data generated in this study but not included in the tables of this article may be requested from the corresponding author.
Declaration of conflicting interest
The authors declare that there is no conflict of interest.
Funding
This work was supported by the German Federal Ministry of Education and Research (BMBF) (grant number 01GY1905) & Software AG Foundation.
