Developing a mobile health app for chronic illness management: Insights from focus groups

Abstract

Background

Mobile applications have the potential to assist patients and caregivers in managing chronic diseases, but current offerings are typically limited to a single disease or only focused on patients. A multifaceted app could expand the user base and provide more comprehensive support for patients who usually suffer from diverse chronic conditions at the same time and their caregivers.

Objectives

This study aimed to design, develop, and evaluate a mobile application that could aid chronic patients and their caregivers in managing the range of chronic conditions.

Methods

Focus group meetings were conducted with patients and caregivers to identify their needs, which were translated into app functionalities. The final version of the app was evaluated through a survey and analysis of the most used features.

Results

Five focus group (FG) meetings were held, involving 39 patients and 22 caregivers. The app included medication aids, self-monitoring aids, activation reminders, messages for caregiver shifts, remote monitoring, medicine management, and informative videos. Testing by 65 patients revealed that medication reminders (96.8% positive responses), medicine information (96.8%), messages for caregiver shifts (90.3%), and ease of use (80.6%) were the most valued aspects. The most frequently used feature was the health data record check.

Conclusions

A multifaceted mobile application can address the needs of a variety of chronic patients and their caregivers, while still being easy to use. The app's most used functionalities were aligned with the needs expressed in the FG, which reflects the success of this study and suggests that future research could benefit from incorporating similar user-centered design methodologies.

Keywords

mHealth focus groups chronic diseases patients caregivers monitoring

Introduction

According to the World Health Organization (WHO),¹ noncommunicable diseases (NCDs) are responsible for 71% of all global deaths, amounting to 41 million deaths each year. Most of these deaths are attributed to chronic diseases, such as cardiovascular diseases (17.9 million deaths per year), respiratory diseases (4.1 million deaths per year), and diabetes (1.5 million deaths per year). These findings are consistent with those of a recent study,² which found that NCDs accounted for 72.3% of global deaths in 2016 across 195 locations. The economic burden of chronic diseases is also significant, with estimated annual expenses of $3.7 trillion.³

Effective self-management is essential for individuals with chronic diseases, as they must take responsibility for minimizing the impact of their illnesses without the support of health professionals.⁴ Considering that people over 65 often accumulate several chronic conditions concurrently, the relevance of self-management becomes particularly important for all healthcare systems. As pointed out by the study presented in⁵ with primary care patients suffering multiple chronic diseases, self-management skills can be improved by promoting the patient-provider relationship through personalized approaches.

Digital tools are particularly effective in fostering such personalization, facilitating communication between patients and healthcare providers, and providing further benefits in areas such as medication adherence. This aspect is a recurring challenge among patients with a wide range of chronic diseases, with nonadherence often leading to severe health complications over time. The overview of reviews presented in⁶ analyzes interventions focused on improving adherence to different chronic conditions (diabetes, hypertension, heart condition, stroke, or cognitive impairment) and concludes that some interventions should be prioritized, among them are dose simplification and reminders. In this sense, mobile apps can help in increasing adherence by creating reminders for medication intake, diet, and physical activity. Among other studies in this field, Pérez-Jover et al.⁷ review the improvements in adherence that can be obtained by using mobile apps in several diseases including asthma, heart failure, hypertension, and HIV, concluding that mobile apps help in terms of perceived usefulness and actual efficacy. Similarly, Cheikh-Moussa et al.,⁸ in their review of reviews, analyze the effect of mHealth interventions in patient engagement for non-pediatric chronic cardiometabolic diseases, concluding that technologies incorporating smartphones and reminder functions were the best option in terms of engagement results.

Chronic diseases are usually accompanied by a complex medication schedule, with multiple different medicines and doses. This complexity can, unfortunately, lead to frequent medication errors, such as forgetting doses, doubling doses, or confusing medications. Several studies have sought to understand the prevalence and severity of these errors, underlining the critical need to mitigate them. A descriptive study by Mira et al.,⁹ which involved interviews with 199 patients and 60 professionals, found that 29.5% of patients and 70% of professionals reported frequent medication errors. The most recurrent error was patients taking the wrong medication. A cross-sectional study¹⁰ focusing on elderly patients on polypharmacy (n = 265) revealed these patients’ lack of understanding about necessary precautions and appropriate methods to store their medications. Furthermore, a review presented in¹¹ stated that the incidence of medication errors varied between 19% and 59%. Common errors included incorrect dosage, forgetting or mixing up medications, inability to recall indications, or usage of out-of-date or improperly stored drugs. Digital tools, such as electronic pillboxes or health apps, can help reduce such errors.^12–14 In these tools, patients value ease of use and simple interfaces, as concluded in.¹⁵

Caregivers, defined as people who provide care to an adult or child with special needs, play a critical role in the management of chronic diseases. In the study presented in,¹⁶ it is estimated that, as of 2020, there were 53 million caregivers in the USA and that most of them (around 47 million) were providing unpaid care (they were family caregivers or FCGs), the rest being professional caregivers. The burden of caregiving can negatively impact the health of FCGs, reducing their ability to provide effective care.¹⁷ Digital tools can also help reduce the burden of caregiving, as demonstrated in the systematic review conducted by Sala-Gonzalez et al.,¹⁸ which evaluated mobile apps designed specifically for FCGs.

Combining the previous points, it would be interesting to simultaneously satisfy the needs of patients and caregivers via digital tools. The challenge is to cover the needs of patients and caregivers while keeping a simple, usable interface. In addition, the same tool should be valid for different diseases and various patient demographics, including pediatric, adult, and elderly individuals.

Most previous approaches have been focused on specific diseases or patient groups. On the one hand, concerning pediatric patients, previous studies on the use of digital tools to help patients and caregivers include interventions for specific diseases like muscular dystrophies,¹⁹ asthma,²⁰ atopic dermatitis,²¹ cystic fibrosis,²² and interventions focused on pediatric emergencies.²³ On the other hand, concerning adult and older patients, previous research includes different proposals of digital tools for dementia or severe mental illnesses,^24–26 cirrhosis,²⁷ and Parkinson disease²⁸ or on patients with a left-ventricular assist device.²⁹ Finally, although not exactly in the scope of the present study, there are also digital tools proposed for helping patients and caregivers with non-chronic diseases, mostly focused on cancer.³⁰

In addition to helping patients and caregivers, digital tools can also aid in patient monitoring. For example, the ValCrònic study³¹ remotely monitored in a real context 521 patients for 1 year, resulting in significant improvements in patient health measures such as weight, blood pressure, and glycated hemoglobin. The PONIENTE study³² demonstrated a 0.09 quality-adjusted life-year (QALY) improvement in 1 year for telemonitored patients (82 participants), with an estimated 58% reduction in treatment costs compared to the control group. The TELBIL project³³ also showed improvements in patient quality of life and cost reduction. Recently, the Tholomeus project³⁴ proved effective for remote monitoring of various diseases, including arterial hypertension, heart disease, chronic obstructive pulmonary disease, and obstructive sleep apnea.

The work presented in this paper tries to combine patient and caregiver tools with patient monitoring. The inspiration for this work came from the ValCrònic study,³¹ which focused on remote monitoring using specialized hardware, and the Proprese project,³⁵ which aimed to personalize and monitor patient objectives. The goal was to develop the ATLAS mobile app, which incorporates remote monitoring (similar to ValCrònic, but using smartphones instead of specialized hardware) and personalized objective follow-up (similar to Proprese, but with remote follow-up instead of in-person). This point is relevant, as ATLAS prioritizes the active participation of patients or their caregivers in the control of their health. The ATLAS app also includes self-management tools for patients and supportive tools for caregivers. To ensure that the needs of patients and caregivers are properly addressed, the strategy employed involved extracting ideas from focus groups (FG) with patients and caregivers during the app design and development processes. The FG approach is similar to that used in some of the aforementioned projects.^{21,22,23,25,30}

Methods

The design and development of the app were guided by the conclusions extracted from FG held with patients and caregivers. A total of five FG were conducted between April 2, 2019, and October 15, 2019. The design and reporting of the qualitative FG study were conducted following the consolidated criteria for reporting qualitative research (COREQ).³⁶

Participants were recruited through convenience sampling using the snowball technique through collaboration with a caregivers’ association (CUIVAL, Association of Caregivers and Relatives of Valencia) and four diabetes associations (Valencia, Elda, Villena, and Muro de Alcoy). Inclusion criteria: people suffering from diverse chronic conditions or caregivers of someone with a chronic disease. Within the diabetes associations, patients suffering secondary chronic diseases apart from diabetes were prioritized. Exclusion criteria: patients receiving medical care outside the public health system, patients with ongoing claim processes, and participants unwilling to sign the informed consent. Recruitment also considered the gender balance in the groups.

Each FG session lasted approximately 90 minutes, adhering to a structured guide featuring predetermined key questions (this guide is provided as supplementary material, both in its original Spanish version and translated to English). Every session adhered to the following format:

Introduction of the moderator(s) and the research team

Explanation of the study's purpose and objectives

Request for consent to record the session, followed by participants signing informed consent forms

Participants’ round of introductions, during which each patient or caregiver detailed their diseases and caregiving strategies

Group discussion, guided by the pre-established structured guide

Recordings of the sessions were transcribed manually. The analysis of the transcription was carried out through a self-developed procedure, structured in four successive steps.

Firstly, the ideas expressed in each FG were analyzed to check their suitability for being implemented in the app. Whenever feasible, the app was updated with new or modified functionalities prior to conducting the subsequent FG. Decisions regarding the feasibility and suitability of ideas were reached through consensus among a team comprising three of the paper's authors, including the project lead and two software developers. In cases of disagreement, decisions were made by a majority vote.

Secondly, we performed a triangulation analysis of transcribed information from all the FG. This analysis aimed to identify identical or similar ideas and highlight differences in proposals both within the same group and among different groups. This approach allowed us to identify the most frequently repeated proposals and ideas, indicating the highest level of acceptance among participants. Information within each category was classified based on spontaneity (how often the same idea was independently repeated) and consistency (whether the same idea was repeated across different groups). We selected verbatim descriptions from each unit of analysis based on their relevance, frequency, or when there was a high level of agreement among participants.

Thirdly, we conducted an analysis that consolidated all transcribed information related to how the app should be used. Three researchers were involved in this process, transforming and combining transcribed information into mutually exclusive categories. Initially, the first researcher performed categorization based on the FG script and transcriptions of each session. Subsequently, the second researcher examined the level of agreement regarding the proposed categorization by the first reviewer, resulting in the addition or elimination of categories as needed. In cases of disagreement, a third reviewer determined the most appropriate categorization for the verbatim data. This approach ensured that all proposals and ideas regarding the app's usage were identified and considered, to achieve user-friendly functionality aligned with the needs of end-users.

Fourthly, given the need to tailor the tools to the potential distinct needs of patients and caregivers, we specifically considered the similarities and differences between patients and caregivers in their contributions during the group sessions. The information obtained from the contributions of both patients and caregivers was triangulated (cross-validation) to analyze the degree of consistency (repeated proposals from both groups) of the ideas put forward. To facilitate this analysis, we assigned values: one for an idea repeated in one group, two for an idea repeated in two groups, and so forth, up to a maximum of five groups.

The final app was ready in December 2019 and was made available to all participants through their associations, who could also offer the installation link to other interested patients or caregivers. After three months of app usage (March 2020), the participants were contacted again to evaluate the app. Due to the pandemic and lockdowns, the evaluation could not be carried out in person. Instead, we carried out an online cross-sectional survey study from March to April 2020.

The ad hoc survey was completely self-developed, based on other surveys used in previous studies on mobile app usage,¹⁴ and consisted of two main sections. The first section assessed the app as a whole, including usability aspects (ease of use, design, font sizes, button accessibility) and also asked the users whether they would use the app in the future and whether they would recommend it to other patients or caregivers. The second section focused on specific app aspects: ease of access to the caregivers’ area, audibility of the alarms, the usefulness of different functionalities (medication reminders, medicine information, advice videos, information sharing between caregivers, error reporting), and usefulness and simplicity of app charts. The scale for all the questions in these two sections consisted of two response options (yes/no). To conclude the survey, satisfaction with the mobile application was rated on a scale of 0 to 10, where the highest score represented maximum satisfaction. Additionally, participants were allowed to provide qualitative comments about the mobile application. Prior to its online submission, the survey was presented to two caregivers to assess the readability and comprehension of the questions. The final survey questionnaire is provided as supplementary material, both in Spanish (original version) and in English.

Our main hypothesis was that an app structure with two different areas (one for patients and another one for their caregivers), as well as the option to access both areas for nondependent patients, after the inclusion of the ideas expressed in the FG, could fulfill all different needs and still be easy to use. The ease of use was one of the key points in the final evaluation survey.

Figure 1 outlines the study's flowchart. The initial version of the app only had fundamental functionalities: health status monitoring, medication management, and alarms, and this version was introduced in the first FG. After each FG session, participants’ feedback was analyzed, and where applicable, translated into new or modified app features, leading to the development of an updated app version. This revised version was then introduced in the subsequent FG. This cycle was repeated until all five FGs had taken place. At this point, all FG results were combined for further analysis and guided the last app improvements. Subsequently, the final version of the app was distributed to all participants, who were asked to use it for 3 months. At the end of this period, participants were requested to complete an evaluation survey.

Figure 1.

Study flowchart.

Results

Results obtained in FG

Results from the FG revealed valuable insights into the needs and preferences of patients and caregivers. The groups were organized as follows:

FG1: Focus group held with the association of caregivers and relatives of Valencia (CUIVAL, Asociación de Cuidadores y Familiares de Valencia) (April 2, 2019)

FG2: Focus group held with the diabetes association of Villena (July 4, 2019)

FG3: Focus group held with the diabetes association of Elda (July 11, 2019)

FG4: Focus group held with the diabetes association of Muro de Alcoy (July 26, 2019)

FG5: Focus group held with the diabetes association of Valencia (October 15, 2019).

A total of 39 patients and 22 caregivers participated in these group sessions. The sex of participants was balanced (49.1% female), and their ages ranged from 14 to 81 years (average 56.98, std. 13.49). The most common caregiver-familiar relationship was mothers taking care of their children and sons or daughters taking care of one of their parents. Table 1 provides details on the attendees of each FG. In this table, the column “dyads” refers to the number of occurrences of a patient and his/her caregiver attending the same FG. Such circumstances only occurred twice (one dyad in FG2 and one dyad in FG4). No further comparative analysis of responses was carried out (dyads vs non-dyads), as the statistical significance would have been extremely low due to the reduced number of dyads. The column “pediatric disease” shows the number of caregivers of pediatric patients (a total of nine caregivers who attended FG2, FG3, and FG5). Globally, pediatric diseases represented 14.7% of all participants.

Table 1.

Participants in the focus groups (FG).

Focus group	Attendees	Female	Male	Patients	Caregivers	Dyads	Pediatric disease
FG1	8	4 (50%)	4 (50%)	1 (12.5%)	7 (87.5%)	0 (0%)	0 (0%)
FG2	18	8 (44%)	10 (56%)	12 (66.7%)	6 (33.3%)	1 (5.6%)	5 (27.8%)
FG3	21	10 (48%)	11 (52%)	17 (81%)	4 (19%)	1 (4.8%)	1 (4.8%)
FG4	6	3 (50%)	3 (50%)	5 (83.3%)	1 (16.7%)	0 (0%)	0 (0%)
FG5	8	5 (62.5%)	3 (37.5%)	4 (50%)	4 (50%)	0 (0%)	3 (37.5%)
Total	61	30 (49.1%)	31 (50.9%)	39 (63.9%)	22 (36.1%)	2 (3.3%)	9 (14.7%)

Table 2 aggregates the diseases of attending patients (patients taking care of themselves) and of patients being cared for by the caregivers who attended (patients with caregivers). The main and secondary diseases of all patients are included in the table, with a total of 15 secondary diseases. Counts and percentages are computed as the number of patients suffering from a certain disease. Globally, type II diabetes was the most prominent disease (59.0% of patients), followed by type I diabetes (21.3%), hypertension (18.0%), dementia or Alzheimer (13.1%), rheumatic diseases (8.2%), and functional diversity (4.9%). Please refer to Table 2 for further details and a summary of these findings.

Table 2.

Diseases of patients.

	Patients taking care of themselves	Patients with caregivers	Total
Number of patients	39	22	61
Number of secondary diseases	11	4	15
Type II diabetes	30 (76.9%)	6 (27.3%)	36 (59.0%)
Type I diabetes	5 (12.8%)	8 (36.4%)	13 (21.3%)
Hypertension	9 (23.1%)	2 (9.1%)	11 (18.0%)
Dementia/Alzheimer	0 (0.0%)	8 (36.4%)	8 (13.1%)
Rheumatic diseases	5 (12.8%)	0 (0.0%)	5 (8.2%)
Functional diversity	1 (2.6%)	2 (9.1%)	3 (4.9%)

Analysis of patients’ comments

Patients’ comments were categorized into five mutually exclusive topics:

Primary care (PC) and health professionals’ (HP) issues (36% of patients)

Medication and treatment (33%)

Diet-related issues (10.25%)

Need for psychological support (2.5%)

Need for financial assistance (2.5%)

The complete list of needs expressed by patients, along with the topic it is categorized in, is shown in Table 3, with spontaneity and consistency measures as defined in the Methods section. Additionally, the table indicates whether each topic can be addressed by an app functionality.

Table 3.

Needs expressed by patients in the focus groups (FG) (marked with † those susceptible of being addressed through an app functionality). No topics were detected in FG1 as participants were mostly caregivers.

Need expressed	Spontaneity					Consistency
Need expressed	FG1	FG2	FG3	FG4	FG5	Consistency
PC and HPs’ issues
Better training of health professionals		1	1			2
More information at the time of diagnosis		1	1			2
More information on primary care			3		1	2
Better management of human resources			1			1
Comprehensive controls and reviews in primary care			2			1
Medication and treatment
† Remembering to take medication		1	4		3	3
Alleviate the side effects of medication				2		1
Change the format of the pills (pharmaceutical companies)			2			1
† Help to combine different treatments			1			1
† Adapting the treatment to individual needs		1			1	2
† More information about the treatment			1			1
† Symptom management		1				1
Diet-related issues
† Control feeding		1		2	1	3
Need for psychological support
† Psychological support				1		1
Need for financial assistance
Economic aid			1			1

Some verbatims of the patients’ comments are shown below as an example, translated into English. A complete list (in Spanish) can be obtained by contacting the authors of this study:

Male, diabetes: “Lately I’m forgetting my medication in the morning” (category: medication and treatment).

Male, diabetes: “Sometimes I am taking the pills that I need, but someone calls me and then I no longer remember if I have taken them or not” (category: medication and treatment).

Male, diabetes: “Pharmaceutical companies change the format of pills and older people go crazy. I need to ask: what is this new medicine for?” (category: medication and treatment).

Female, hypertension: “I had to stop taking the medication because it made me lose a lot of weight” (category: medication and treatment).

Male, diabetes: “What is most difficult for me is controlling my diet. The anxiety of celebrations, going out with friends… and eating things you shouldn't” (category: diet following issues)

Analysis of caregivers’ comments

Discussions with caregivers were categorized into eight mutually exclusive topics:

Need for psychological help (59% of caregivers)

Primary care (PC) and health professionals’ (HC) issues (41%)

Coping with patient symptoms (36.5%)

Need for information and communication during caregiver shifts (27.3%)

Medicine management (23%)

Diet following issues (13.6%)

Need for economic aid (13.6%)

Lack of adaptation in schools (13.6%)

Table 4 shows the complete list of needs expressed by caregivers, along with the topic it is categorized in. Analyzing the needs expressed, the only specific requirement for pediatric patients was “adapt educational centers for kids with diseases.” As caregivers of pediatric patients were represented in the FG (36.4%), this fact may reflect that some needs are common to all kinds of caregivers, either caring for adult patients or pediatric patients. Additionally, the table indicates whether each topic can be addressed by an app functionality.

Table 4.

Needs expressed by caregivers in the focus groups (FG) (marked with † those susceptible of being addressed through an app functionality). No topics were detected in FG4 as participants were mostly patients.

Need expressed	Spontaneity					Consistency
Need expressed	FG1	FG2	FG3	FG4	FG5	Consistency
Need for psychological help
† Psychological support for caregivers	1					1
† Psychological support for patients		2			2	2
Solve emotional overload	3					1
Manage feelings of guilt	2					1
Manage difficulties in relationships with the rest of the family	2					1
Having to respect the decisions of the patient	3					1
† Promote disease awareness in the patient	2	1			2	3
† Provision of support to the patient	2					1
Increase the knowledge of society about the disease					1	1
PC and HPs’ issues
Better training of health professionals					2	1
More information at the time of diagnosis		2			2	2
More attention from healthcare professionals		2				1
Coping with patient's symptoms
Compensate for the symptoms of the patient	3					1
Oversee the needs of the patient	2	1				2
Caregiver shifts
† Sharing information with other caregivers	2	1	1			3
† Need for coordination with other caregivers	2					1
Medicine management
† Control treatment adherence of the patient		3				1
† Manage the medication of the patient	2					1
† More information about patient treatment			1			1
Diet following issues
† Control feeding of patient		2	1			2
Need for economic aids
Economic aid for patients	3					1
Lack of adaptation in schools
Adapt educational centers for kids with diseases					2	1

Some verbatims of the caregivers’ comments are shown below as an example, translated into English. A complete list (in Spanish) can be obtained by contacting the authors of this study:

Male, hypertension: “My mother usually forgets taking her pill and we have set the alarm on her mobile. She doesn't know how it works, but she knows that when it rings, she must take the pill” (category: medication and treatment).

Female, dementia: “It is very difficult when the relative is not aware of his disease” (category: need for psychological support).

Female, dementia: “It is assumed that the family has to help and if you don't do it, you feel guilty, a bad father or a bad son” (category: need for psychological support).

Male, dementia: “We have problems managing medication among all caregivers. Sometimes the patient receives double medication due to lack of communication” (category: need for information and communication in caregiver shifts).

Global needs and translation to app functionalities

When comparing the needs expressed by patients and caregivers, there were areas of agreement in their interests, such as treatment adherence, disease information, comprehensive care from health professionals, financial support, and psychological support. However, patients placed more emphasis on receiving better healthcare, while caregivers placed more emphasis on psychological support.

Table 5 shows the topics from previous Tables 3 and 4 that were considered for app design, along with the functionalities added or modified in the app to address each topic.

Table 5.

App functionalities added or modified according to the needs extracted from focus groups (FG).

Need expressed	App functionality
Remembering to take medication	Medication alarm and pillbox stock management.
Help to combine different treatments	Uniformity in all treatment elements: medication, exercise, diet (similar alarms and monitoring).
Control feeding	Simple daily diet registration (good, average, bad).
Adapting the treatment to individual needs	Personalized goals for exercise, diet, and health measures.
More information about the treatment	Informative videos for patients and caregivers.
Symptom management	Patients are provided with recommendations when their measures are out of range.
Psychological support for patients and caregivers	Direct access to specialized Facebook groups.
Provision of support to the patient	Direct access to specialized Facebook groups.
Control treatment adherence of the patient	Basic and detailed evolution monitoring for the patients. Reports are periodically sent to caregivers.
Manage the medication of the patient	Pillbox stock management and medicine help functionality.
Promote disease awareness in the patient	Gamification for increasing engagement.
Sharing information with other caregivers	Caregiver notes (text, audio, and images).
Need for coordination with other caregivers	Common calendar.

Final app design

The final design of the ATLAS app consisted of two main areas: a simple area with just a few functionalities for patient access and a more complex area only for caregivers (or for those patients taking care of themselves).

The set of functionalities included in the patient and caregiver areas were the same in all cases, with independence of the patient's diseases, in order to avoid configuration steps. These functionalities are described below, with an explanation of their target users.

Starting with the patients’ area, or the simple area with an easier interface, these are the functionalities included:

Alarms functionality, which is structured so that the caregivers configure the alarms (medicine alarms, exercise alarms, or alarms to take health measures) and the patients only receive the alarms at the scheduled time. This functionality can be useful for almost all patients, including early-stage dementia patients. For late-stage dementia, alarms are not recommended, and their caregivers should not configure them.

Status functionality and activity check, which shows the patient's fulfillment of health goals, which were previously configured by the caregivers. Useful for most patients except dementia patients, whose caregivers simply should not configure health goals.

Medicine information, which may be useful for most patients, including early-stage dementia patients, as the information is received as simple-to-understand audios, previously recorded by the caregivers.

Emergency call, which may be useful for all patients, even for late-stage dementia patients.

GPS functionality to track patients’ location, which is also configured by caregivers (if needed) and could be useful for early-stage or late-stage dementia.

The caregiver area had additional functionalities and was supposed to be accessed only by caregivers or by patients who take care of themselves. Focusing on ease of use, this access control was carried out automatically: when an app user did not register any caregivers, it was assumed that the patient was self-sufficient and thus could access all app areas. If the app user declared one or more caregivers, the patient was automatically limited to the simple app area, and the access to the more complex area was locked with an email and password control, so that only the caregivers could access it.

The list of functionalities in the caregiver area included:

Communication between caregivers via logbook messages with text, audio, and images

GPS monitoring setup

Instructional videos for the most common caregiver tasks

Medical facility search (hospitals, drugstores, first care centers)

User management for editing patient, caregiver, or healthcare worker data

App error notification for communication with technical support

App information

Patients were remotely monitored by their healthcare professionals, who received daily reports on the patient's health status. The main screens of the app are shown in Figure 2 (patient access area) and Figure 3 (caregiver access area). Figure 4 provides a detailed view of the initial app screen.

Figure 2.

Patient access area.

Figure 3.

Caregiver access area.

Figure 4.

Initial screen of the app.

Concerning health data monitoring, the app was designed to cope with patients entering their data manually as well as with patients making use of Bluetooth or WIFI-enabled devices (provided the devices were synchronized with their Google Fit accounts).

For non-connected devices, the app included a simple interface to enter data manually. Transparently to the user, the app automatically wrote these data in the Google Fit account of the user through the Google Fit APIs. For connected devices, the users were instructed to synchronize them with their Google Fit accounts. Steps and distance measures were, obviously, automatically synchronized from the patients’ smartphones.

As a result, all kinds of data were stored homogeneously in the Google Fit account of the users. No other external databases were used, to assure the privacy of health data. Figure 5 summarizes the information flow between connected and non-connected devices, the app, and the Google Fit cloud storage.

Figure 5.

Health data gathering, storage, and management.

The app kept evolving after the end of the experimental period. The screenshots shown in Figures 2 to 4 correspond to the version used for the experiment (Atlas app, V1.14). Further evolution was focused on patient-provider communication. Such communication, during the experiments, was based on periodic emails where professionals received the health data of the patients they cared for, as well as the fulfillment of their health goals. More in detail, each patient (or caregiver for dependent patients) could establish a set of professionals who will have access to their health data, which may include doctors, nurses, or other professionals. Future work, currently being tested on newer app versions, includes the change to a different patient-provider communication, carried out through a web application where professionals will be able to monitor all their patients’ data continuously, receive alerts, and even modify individual health goals.

App usage data and assessment of users’ experience

Overall, a total of 65 patients used the ATLAS app (24 males, 35 females, six did not specify gender) and kept the app installed for an average of 111.37 days. Table 6 displays the usage of the app's main functionalities, including medicine scan for information, health data measures, and alarms. Functionality usage is measured as the percentage of app users who accessed each functionality at least once. Despite medicine management being one of the most requested needs during the FG, the most widely used functionality during app usage was checking health data records (84.6% of users accessed their health data records), followed by alarms and medicine scans for information (41.5% and 18.5% of users, respectively). The “nonautomatic health data uploads” category refers to all data that were not automatically sent to the Google Fit account of the patients. Typically, steps and distance were sent automatically, but blood pressure, weight, diet, temperature, and other data had to be manually sent. In total, 38.5% of users utilized this functionality. Globally, 12 app users (18.5%) registered as health professional to receive daily updates of their health values. Concerning caregiving, a total of 19 app users (29.2%) registered at least one caregiver, which means that the remaining 46 patients (70.8%) took care of themselves.

Table 6.

Usage of the main app functionalities (percentage of users).

Functionality	% (n)
Health data record check	84.6 (55)
Alarms	41.5 (27)
Nonautomatic health data uploads	38.5 (25)
Medicine scans for information	18.5 (12)
Registered at least one caregiver	29.2 (19)
Health data monitoring by professionals	18.5 (12)

Table 7 provides a detailed overview of the specific items most frequently used by patients in relation to health data, medicine information, and alarms. In all cases, the measures are expressed as the aggregated number of accesses to a specific functionality, among all app users. With respect to medicine scans for information, the most frequently requested information was “what is it for” (54.6% of medicine information requests were directed to this particular information). Regarding health data records, the most frequently viewed information was the number of steps taken (43.9% of checks), followed by weight (17.7%) and diet (11.0%), which is consistent with the importance given to diet-related issues during the FG attended by both patients and caregivers. Among all alarms scheduled (across all users) medicine alarms accounted for a total of 46% of all alarms, while diet measurement alarms were also frequently used (37.3%), as were exercise alarms (16.7%).

Table 7.

Specific item usage (times used).

Medicine info requested (total requests: 44)	% (n)
What is it for	54.6 (24)
How should I take it	13.6 (6)
Other info	31.8 (14)
Health data records checked (total checks: 2592)
Steps walked	43.9 (1139)
Weight	17.7 (460)
Diet following	11.0 (284)
Blood pressure	5.6 (145)
Other health data	21.8 (564)
Alarm type set (total alarms: 496)
Medicine alarm	46.0 (228)
Diet measurement alarms	37.3 (185)
Exercise alarms	16.7 (83)

Out of the 65 users with access to ATLAS, 31 responded to the structured app assessment survey, resulting in a response rate of 47.7%. The aspects most highly valued included the provision of medication information, reminders for medication intake, caregiver note-sharing capabilities, ease of use, and error reporting (see Table 8). Notably, 80.6% of survey respondents agreed with the app's ease of use. This achievement is in line with one of our primary project objectives: to create an application that caters to the needs of both patients and caregivers across various chronic diseases while ensuring user-friendly functionality.

Table 8.

Users’ assessment of the usefulness of ATLAS and its functionalities (N = 31).

Aspects assessed	% (n) ^†
ATLAS is easy to use	80.6 (25)
I like the ATLAS design	71.0 (22)
The font size is sufficient	77.4 (24)
It is easy to press the correct button with the finger	77.4 (24)
The medication reminder is helpful	96.8 (30)
The reminder alarm is audible	58.1 (18)
Information about medications helps take them correctly	96.8 (30)
Advice videos are helpful	77.4 (24)
Charts are simple	77.4 (24)
Charts are informative	67.7 (21)
The error reporting function is interesting	80.6 (25)
Caregiver access is easy	77.4 (24)
The notes function for sharing information among caregivers is helpful	90.3 (28)
I would use ATLAS in the future	64.5 (20)
I would recommend ATLAS to other patients or caregivers	77.4 (24)
ATLAS overall rating ≥ 8 (on a scale from 0 to 10)	67.7 (21)

^†

Percentage and total number of positive answers.

The survey also featured an open-text field for comments, which let us know that the users found the tool to be easy to use and comprehensive. They appreciated the inclusion of resources for providing instrumental and emotional support, such as videos with advice for patients and caregivers, beyond clinical issues. Users found the possibility of synchronizing Google Fit and ATLAS to be a great help, as it allowed for automatic data recording and avoided potential errors from manual data entry. The GPS patient tracking functionality was particularly useful in situations of dependency.

Users also provided suggestions for areas of improvement in the ATLAS app, most of which were related to their specific health conditions. Users with diabetes suggested incorporating sensors that monitor blood sugar levels and allow for real-time charts, alerts for hypoglycemia or hyperglycemia, and adaptation of activity goals based on this clinical parameter. Caregivers of dependent persons recommended the incorporation of diaper status sensors to facilitate timely diaper changes. Individuals with reduced mobility suggested the inclusion of maps indicating the location of accessible places adapted to their condition, such as parks, restaurants, and stores. In terms of functionalities for the caregiver area, users recommended incorporating a record of visits and medical check-ups to keep everyone involved in the patient's care informed. Lastly, users recommended adapting the app for web format.

Discussion

One of the main goals of the ATLAS app design was to cover the needs of patients and caregivers over a variety of diseases. Previous approaches are either focused on patients and caregivers for a specific disease or, on the other hand, are valid for different diseases but do not cover the needs of caregivers.

In the first category, there are previous m-Health apps designed for chronic patients and their caregivers but focused on specific diseases. Examples include apps targeting mental illnesses,^24–26 cirrhosis,²⁷ Parkinson's disease,²⁸ cystic fibrosis,²² pediatric atopic dermatitis,²¹ or pediatric asthma.²⁰

In the second category, there are telemedicine apps and systems valid for a variety of diseases. In particular, those systems focused on patient monitoring, like the recent Tholomeus project,³⁴ which has been applied to diseases like arterial hypertension, heart disease, chronic obstructive pulmonary disease, and obstructive sleep apnea. However, such approaches do not consider the needs of caregivers.

In contrast, the ATLAS design covers different diseases (diabetes, hypertension, and dementia) with common and specific requirements, providing tools for patients and caregivers in a single app. Although combining all these requirements while maintaining ease of use was a challenge, the results were successful (with 80.6% of users considering the ATLAS app easy to use). Therefore, the ATLAS design can serve as a guide for the creation of future multipurpose apps, covering the needs of a broad range of patients and their caregivers.

The ATLAS app design was the result of FG held with patients and caregivers, similar to previous studies for specific diseases. For example, FG were used in³⁰ for the design of apps focused on cancer patients and their caregivers, in²² for cystic fibrosis, in²¹ for atopic dermatitis, and in²⁵ for mental illnesses. Other studies used participatory design, such as the translation app for foreign-language patients presented in³⁷ or the smartphone app for adolescents with anxiety presented in³⁸. However, all these studies addressed specific diseases or tasks, while the ATLAS app covers the needs of patients and caregivers over different diseases.

Our experimental app incorporated a simple data validation system, computing goal fulfillment for those health values with predetermined goals, and presenting the results to both the app user (patient or caregiver) and the professionals via daily reports. We are currently enhancing this feature to include a more complete data validation system that checks all health values for undesirable or dangerous ranges, issuing a variable severity warning to both the app user and the professional if such ranges are detected.

In this sense, to maximize the efficacy of apps like the one proposed in this study, it is essential to integrate them with patients’ medical records. National Health Services should establish open standards to access this information while safeguarding security and privacy. Currently, there are no mechanisms in the Spanish health system allowing third-party access to medical records, other than for retrieving historical data.

Limitations

The initial purpose of the app was to simultaneously satisfy the self-management needs of caregivers and patients, across different chronic diseases. However, the recruitment for the FG was carried out through a caregiver association and four different diabetes associations because other chronic diseases (e.g., hypertension) lack a comparable association structure, to our knowledge. To gain insights about patients with diverse chronic conditions, one of the criteria for the selection of participants within diabetes associations was the presence of additional diseases apart from diabetes. Such a goal was partially achieved, as other diseases were also represented, such as hypertension (18%), dementia (13%), rheumatic diseases (8.2%), or functional diversity (4.9%). However, the prevalence of diabetes may have biased the FG findings.

In relation to caregivers, their needs vary depending on whether they are caring for adult or pediatric patients. We aimed to design an application useful for both caregiver categories, as both were represented in our FG. We recruited a total of 22 caregivers, 8 of whom (36.4%) were caring for children. Nevertheless, the only specific requirement for pediatric caregivers emerging from the FG was to “adapt educational centers for kids with diseases,” which does not directly influence app design or functionality. Therefore, based on the needs expressed during the FG, we assume that most app functionalities are applicable regardless of whether the individual being cared for is an adult or a child. However, a targeted study could be carried out to identify and address the specific needs of pediatric caregivers.

Patient-provider communication was limited to daily emails where professionals (doctors, nurses, or other health professionals) received patient health data and updates on goal achievement. A more complete, bidirectional communication system is currently under development, including a web page where professionals can monitor their patients’ health data continuously and modify individual health goals when needed.

One of the questions asked to the app users in the final evaluation survey was “would you recommend the ATLAS app to other patients or caregivers?”. The possible answers to this question were “yes” or “no.” It would have been more informative to use a 0 to 10 Likert scale for this question, in order to compute a standard indicator of user satisfaction like the Net Promoter Score.³⁹

A primary aim of the project was to satisfy a variety of needs (patients, caregivers, different diseases) while keeping a simple interface. The questionnaire used to evaluate the app measured simplicity but was not designed to measure the capability of the app to fulfill the needs of all kinds of patients and caregivers across multiple diseases. The information extracted from the surveys only provides an aggregate measure of satisfaction with each app’s functionality.

One of the main functionalities of the app is the analysis of patients’ health data in comparison with individual health goals. However, since the health measuring devices were chosen by the users themselves (with most opting for devices they already owned), we could not verify the reliability of their measurements. Even step count data may vary depending on the specific smartphone, smartwatch, or wristband used. Providing all users with standardized and tested devices could improve the reliability of study results.

Further testing with a wider sample, involving more patients, caregivers, pharmacists, and health professionals, could yield additional insights and opportunities for app refinement. An important limitation arose due to the COVID-19 pandemic and the subsequent lockdowns, which made it difficult to continue with the experiments from March 2020 onwards, particularly with app the evaluation phase.

Conclusions

This study demonstrates that a single mHealth app can effectively meet the diverse needs of patients and caregivers with different diseases, including diabetes, hypertension, and dementia. The users’ assessment also showed that the app was easy to use despite its wide range of functionalities. The design methodology based on FG with patients and caregivers proved successful in identifying the most demanded functionalities for the app. The fact that the most used functionalities during the app usage aligned with the needs expressed in the FG indicates the effectiveness of this approach. However, it is noteworthy that access to health data records, which was not initially expressed as a need, emerged as the most widely used functionality during the app usage.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231210662 - Supplemental material for Developing a mobile health app for chronic illness management: Insights from focus groups

Supplemental material, sj-docx-1-dhj-10.1177_20552076231210662 for Developing a mobile health app for chronic illness management: Insights from focus groups by César Fernández, María Asunción Vicente, Mercedes Guilabert, Irene Carrillo and José Joaquín Mira in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076231210662 - Supplemental material for Developing a mobile health app for chronic illness management: Insights from focus groups

Supplemental material, sj-docx-2-dhj-10.1177_20552076231210662 for Developing a mobile health app for chronic illness management: Insights from focus groups by César Fernández, María Asunción Vicente, Mercedes Guilabert, Irene Carrillo and José Joaquín Mira in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076231210662 - Supplemental material for Developing a mobile health app for chronic illness management: Insights from focus groups

Supplemental material, sj-docx-3-dhj-10.1177_20552076231210662 for Developing a mobile health app for chronic illness management: Insights from focus groups by César Fernández, María Asunción Vicente, Mercedes Guilabert, Irene Carrillo and José Joaquín Mira in DIGITAL HEALTH

Supplemental Material

sj-docx-4-dhj-10.1177_20552076231210662 - Supplemental material for Developing a mobile health app for chronic illness management: Insights from focus groups

Supplemental material, sj-docx-4-dhj-10.1177_20552076231210662 for Developing a mobile health app for chronic illness management: Insights from focus groups by César Fernández, María Asunción Vicente, Mercedes Guilabert, Irene Carrillo and José Joaquín Mira in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors would like to express their sincere appreciation to the diabetes associations of Valencia, Elda, Villena, and Muro de Alcoy, as well as the association of caregivers and relatives of Valencia (CUIVAL) for their invaluable support and contribution to this research project.

Author contributions

All authors contributed to the conceptualization and methodology of this study. MAV and CF oversaw the app development and data analysis. All authors contributed to the writing of this manuscript, including the original draft preparation and subsequent review and editing.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Committee of the Sant Joan University Hospital in Alicante (reference 18/324, June 27th, 2018). The study was registered with ClinicalTrials, reference NCT04159558.

Funding

This study is part of the research of the Excellence Groups PROMETEO/2017/173 and PROMETEO/2020/061 financed by the “Conselleria de Innovación, Universidades, Ciencia y Sociedad Digital” from Generalitat Valenciana. During the development of this manuscript, J.J.M. enjoyed an intensified research activity contract from the Carlos III Health Institute (reference INT22/00012).

Generalitat Valenciana, Instituto de Salud Carlos III, (grant number PROMETEO/2017/173, PROMETEO/2020/061, INT22/00012).

Guarantor

José Joaquín Mira (J.J.M.)

ORCID iD

María Asunción Vicente

Supplemental material

Supplemental material for this article is available online.

References

Noncommunicable diseases [Internet] [cited on 16 June 2023]. Available from: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases

GBD 2016 Causes of Death Collaborators. Global, regional, and national age-sex specific mortality for 264 causes of death, 1980-2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet 2017; 390: 1151–1210.

O'Neill Hayes

Gillian

. Chronic disease in the United States: A worsening health and economic crisis. [Internet] [cited on 16 June 2023]. Available from: https://www.americanactionforum.org/research/chronic-disease-in-the-united-states-a-worsening-health-and-economic-crisis/.

Guerreiro

Strawbridge

Cavaco

, et al. Development of a European competency framework for health and other professionals to support behaviour change in persons self-managing chronic disease. BMC Med Educ 2021; 21: 87.

Brenk-Franz

Strauß

Tiesler

, et al. Patient-provider relationship as mediator between adult attachment and self-management in primary care patients with multiple chronic conditions. J Psychosom Res 2017; 97: 131–135.

Anderson

Nuckols

Coles

, et al. A systematic overview of systematic reviews evaluating medication adherence interventions. Am J Health Syst Pharm 2020; 77: 138–147.

Pérez-Jover

Sala-González

Guilabert

, et al. Mobile apps for increasing treatment adherence. Systematic review. J Med Internet Res 2019; 21: e12505.

Cheikh-Moussa

Mira

Orozco-Beltran

. Improving engagement among patients with chronic cardiometabolic conditions using mHealth: critical review of reviews. JMIR Mhealth Uhealth 2020; 8: e15446.

Mira

Ortiz

Lorenzo

, et al. Oversights, confusions and misinterpretations related to self-care and medication in diabetic and renal patients. Med Princ Pract 2014; 23: 246–252.

10.

Mira

Martínez-Jimeno

Orozco-Beltrán

, et al. What older complex chronic patients need to know about their everyday medication for safe drug use. Expert Opin Drug Saf 2014; 13: 713–721.

11.

Mira

Lorenzo

Guilabert

, et al. A systematic review of patient medication error on self-administering medication at home. Expert Opin Drug Saf 2015; 14: 815–838.

12.

Mira

. Medication errors in the older people population. Expert Rev Clin Pharmacol 2019; 12: 491–494.

13.

Mira

Navarro

Botella

, et al. A Spanish pillbox app for elderly patients taking multiple medications: randomized controlled trial. J Med Internet Res 2014; 16: 99.

14.

Mira

Guilabert

Carrillo

, et al. Use of QR and EAN-13 codes by older patients taking multiple medications for a safer use of medication. Int J Med Inform 2015; 84: 406–412.

15.

Carrillo

Guilabert

Pérez-Jover

, et al. Valoración de dos aplicaciones para la autogestión de la medicación en pacientes mayores. Estudio cualitativo [assessment of two applications of medication self-management in older patients. Qualitative study]. Rev Calid Asist 2015; 30: 142–149.

16.

AARP and National Alliance for Caregiving. Caregiving in the United States 2020. Washington, DC: AARP, May 2020. [Internet] [cited on 16 June 2023]. Available from: https://doi.org/10.26419/ppi.00103.001

17.

Baik

Song

Tark

, et al. Effects of physical activity programs on health outcomes of family caregivers of older adults with chronic diseases: A systematic review. Geriatr Nurs 2021; 42: 1056–1069.

18.

Sala-González

Pérez-Jover

Guilabert

, et al. Mobile apps for helping informal caregivers: a systematic review. Int J Environ Res Public Health 2021; 18: 1702. Published 2021 Feb 10.

19.

Sobierajska-Rek

Mański

Jabłońska-Brudło

, et al. Establishing a telerehabilitation program for patients with Duchenne muscular dystrophy in the COVID-19 pandemic. Wien Klin Wochenschr 2021; 133: 344–350.

20.

McWilliams

Reeves

Shade

, et al. Patient and family engagement in the design of a mobile health solution for pediatric asthma: development and feasibility study. JMIR Mhealth Uhealth 2018; 6: e8849.

21.

Griva

Koh

, et al. Creating a smartphone app for caregivers of children with atopic dermatitis with caregivers, health care professionals, and digital health experts: participatory co-design. JMIR Mhealth Uhealth 2020; 8: e16898.

22.

Rutland

Bergquist

Hager

, et al. A mobile health platform for self-management of pediatric cystic fibrosis: qualitative study of adaptation to stakeholder needs and integration in clinical settings. JMIR Format Res 2021; 5: e19413.

23.

Ehrler

Siebert

Rochat

, et al. Connecting parents to a pediatric emergency department: designing a mobile app based on patient centred care principles. Stud Health Technol Inform 2017; 244: 13–17.

24.

Than

Nguyen

, et al. Smartphone app-based intervention for reducing stress, depression, and anxiety in caregivers of people with dementia in Vietnam: study protocol for a pilot randomized controlled trial. DIGITAL HEALTH 2023; 9:20552076231163786. doi:10.1177/20552076231163786

25.

Sinha Deb

Tuli

Sood

, et al. Is India ready for mental health apps (MHApps)? A quantitative-qualitative exploration of caregivers’ perspective on smartphone-based solutions for managing severe mental illnesses in low resource settings. PloS One 2018; 13: e0203353.

26.

Forma

Chiu

Shafrin

, et al. Are caregivers ready for digital? Caregiver preferences for health technology tools to monitor medication adherence among patients with serious mental illness. DIGITAL HEALTH 2022; 8: 20552076221084472. doi:10.1177/20552076221084472

27.

Shaw

Acharya

Albhaisi

, et al. Subjective and objective burden on providers from a multicenter app-based study of patients with cirrhosis and caregivers. Hepatol Commun 2023; 7: e0030.

28.

Bernardini S, Cianfrocca C, Maioni M, et al. A mobile app for the remote monitoring and assistance of patients with Parkinson's disease and their caregivers. In: 2018 40th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), July 2018, pp.2909–2912. Honolulu, HI: IEEE.

29.

Casida

Aikens

Craddock

, et al. Development and feasibility of self-management application in LVADs. ASAIO J (Am Soc Artif Intern Organs: 1992) 2018; 64: 159.

30.

Huelster

Zemp

Turner

, et al. Mobile postoperative symptom intervention tool and biometric monitoring after radical cystectomy: pilot study evaluating feasibility, usability, and potential utility. J Urol 2023; 209: 410–421.

31.

Orozco-Beltran

Sánchez-Molla

Sanchez

, et al. ValCrònic Research Group. Telemedicine in primary care for patients with chronic conditions: the ValCrònic quasi-experimental study. J Med Internet Res 2017; 19: e400. Published 2017 Dec 15.

32.

Lopez-Villegas

Catalan-Matamoros

Robles-Musso

, et al. Cost-utility analysis on telemonitoring of users with pacemakers: the PONIENTE study. J Telemed Telecare 2019; 25: 204–212.

33.

Martín-Lesende

Orruño

Bilbao

, et al. Impact of telemonitoring home care patients with heart failure or chronic lung disease from primary care on healthcare resource use (the TELBIL study randomised controlled trial). BMC Health Serv Res 2013; 13: 18.

34.

Omboni

Campolo

Panzeri

. Telehealth in chronic disease management and the role of the internet-of-medical-things: the Tholomeus® experience. Expert Rev Med Devices 2020; 17: 659–670.

35.

Orozco-Beltran

Ruescas-Escolano

Navarro-Palazón

, et al. Effectiveness of a new health care organization model in primary care for chronic cardiovascular disease patients based on a multifactorial intervention: the PROPRESE randomized controlled trial. BMC Health Serv Res 2013; 13: 93.

36.

Tong

Sainsbury

Craig

. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec; 19: 349–357. Epub 2007 Sep 14. PMID: 17872937.

37.

Noack

Schulze

Müller

. Designing an app to overcome language barriers in the delivery of emergency medical services: participatory development process. JMIR Mhealth Uhealth 2021; 9: e21586. Published 2021 Apr 14.

38.

Newton

Bagnell

Rosychuk

, et al. A mobile phone-based app for use during cognitive behavioral therapy for adolescents with anxiety (MindClimb): user-centered design and usability study. JMIR Mhealth Uhealth 2020; 8: e18439. Published 2020 Dec 8.

39.

Reichheld

. The one number you need to grow. Harv Bus Rev 2003; 81: 46–55.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB