Sage Journals: Discover world-class research

Abstract

Objective

The aim of the study was to develop an app to improve patients’ adherence to therapy for osteoporosis and to test its usability.

Methods

In Phase I, the app functions needed to improve medication adherence were identified through a focus group with six patients with osteoporosis and a joint interview with two bone specialists. The app prototype was then developed (Phase II) and refined after its feasibility testing (Phase III) for 13–25 days by eight patients. Finally, the app underwent usability testing (Phase IV) for 6 months by nine other patients. The mHealth App Usability Questionnaire (MAUQ) was used to collect the assessment of the app by the 17 patients.

Results

The final version of the app provided information on osteoporosis, allowed patients to contact the bone specialist for an additional consultation, and generated a reminder for taking medications accompanied by feedback on adherence. The assessment of the app was positive but evaluations differed between the feasibility and usability testing, with the former displaying a significantly (p ≤ .05) better assessment across all MAUQ items.

Conclusions

In this study, we tested an app for improving adherence to medical therapies in patients with osteoporosis. The usability testing revealed a lower “patient-centered” performance of the app as compared to that observed during the feasibility phase. Future developments of the study include increasing the testing cohort and adding a technical support during the usability testing.

Keywords

App mHealth anti-osteoporotic therapies adherence participatory design

Introduction

Osteoporosis is a chronic bone disease that consists of a deterioration in bone structure and impairment in bone strength, leading to an increased fracture risk. The prevalence of osteoporosis worldwide in 2021 was estimated as 23.1% for women and 11.7% for men.¹ In the future, the incidence of osteoporosis is expected to grow with the aging of the population, with an increase by 240% in women and 310% in men by 2050. This trend will make the burden and challenges linked to the care of this chronic condition even more relevant.²

Osteoporosis leads to an increased risk of fractures, especially hip fractures. Notably, it has been calculated that worldwide there is a fracture connected to osteoporosis every three seconds.³ In Europe, more than 23 million people are at risk of this kind of fracture and specifically 22.1% of women over 50 years old and 6.6% of men in the same age group.⁴ The disability-adjusted life years (DALYs) lost to osteoporosis are greater than those lost to cancer, apart from lung cancer,⁵ and fractures have been shown to increase the risk of mortality and morbidity.⁶ Hip fractures lead to a 30% mortality rate at 1 year but also negatively affect the quality of life of osteoporotic patients, with 53% of them dramatically losing independence and autonomy.⁷ The economic impact of osteoporosis is also very significant and, according to the International Osteoporosis Foundation, 47.4 billion Euros will be spent in Europe in 2030, mostly due to hospitalization after fractures.⁸

Controlled-randomized clinical trials provided consistent evidence that bone-active drugs, that are currently available, such as oral bisphosphonates, are effective in improving skeletal health and reducing the risk of fractures in patients with osteoporosis.⁹ However, trials were performed in ideal conditions greatly differing from the real-world scenario, where several interfering factors may influence the response to the therapies.¹⁰ Drug compliance and adherence have proved to be important determinants of the real-world effectiveness of osteoporosis medications.¹¹ The main reasons for non-adherence to osteoporosis medications are the lack of trust in their utility and safety, forgetfulness, treatment complexity, and associated costs.^12,13 Moreover, the less patients with osteoporosis are involved in the treatment and care decisions by the bone specialist, the more likely they will not adhere to the drug therapy.¹⁴ This indicates that the capacity to self-manage and maintain a collaborative and close relationship with the bone specialist are fundamental aspects in improving adherence to drug therapy and preventing the progression of osteoporosis. In particular, with respect to bisphosphonates, their effectiveness in reducing fracture risk has been shown to be blunted when patients take less than 80% of the prescribed medications.^15–17 Data show that non-adherence to bisphosphonates is indeed widespread. Primary non-adherence, which consists in the refusal (or delay) to take drugs after 14–20 days from the first prescription, is attested at 37.5%.¹⁸ Secondary adherence, that is, after 6 months, 1 year and 2 years from the first prescription, ranges between 34.8 and 71.3%, 17.7 and 74.8% and 12.9 and 72%, respectively.¹⁹

Digital health²⁰ and mobile health (mHealth) technology, by means of applications (apps), could provide an important contribution to managing osteoporosis and, specifically, improving adherence to drug therapy.²¹ mHealth solutions are also potentially cost-effective, given that they can prevent/reduce visits to the hospital or other healthcare facilities,²² something particularly convenient for patients with osteoporosis, who have often difficulty of movement and mobility.

In a systematic review of digital health technologies for the long-term management of osteoporosis, 32 apps targeted at patients or bone specialists were identified and analyzed.²³ Apps, for the most part, provided tools for self-management and fracture risk assessment, and advice on nutrition and physical activity. Common features of these apps were diet and exercise monitoring, reminders for vitamin D intake, and data sharing or export functions. Interestingly, none of the apps were aimed at improving the adherence of patients with osteoporosis to medications such as oral bisphosphonates.

Thirteen studies included in the review were randomized controlled trials and provided means for evaluating the effectiveness of the respective apps. The main endpoints used in these studies were app usability, self-reported vitamin D intake, and dietary calcium intake. These studies reported significant improvements for patients with osteoporosis on some outcomes such as pain, disability, and physical functionality, but no significant improvement on others, such as well-being, physical activity, anxiety, fatigue, and calcium and vitamin D intake. A caveat of these results was the generally low number of downloads (below 1000 units) of the apps among patients and the reticence of healthcare professionals to use digital tools, which suggests a general difficulty in integrating this type of solution into routine disease management.²³

The mixed results obtained with the use of apps for the management of osteoporosis may also be due to the fact that, apart from one case, none of the apps were developed with the participation of patients and healthcare professionals, despite evidence that participatory designs can improve the acceptance and effectiveness of apps.²⁴ The only documented case is that of the app named “My Osteoporosis Journey.”²³ This app targeted postmenopausal women newly diagnosed with asymptomatic osteoporosis. It gave information about osteoporosis, collected bone density scans, and provided treatment recommendations. Patients involved in the participatory process reported feeling confident with the information about osteoporosis provided by the app, in comparison to that provided by the Internet and underlined the usefulness of the app in osteoporosis self-management.²⁵

The purpose of this study was to develop, with the participation of patients with osteoporosis and bone specialists, an app aimed at improving patients’ adherence to therapy with a specific focus on adherence to oral bisphosphonates. Beyond the development of the App, the study aimed at providing an initial testing of its usability and acceptability in the perspective of patients.

Methods

Study design and methodologies

The study was prospective and consisted of four phases lasting between November 2021 and March 2023. The study was entirely conducted in Milan (Italy) in the context of the TELIOT project that sees the collaboration of the Politecnico di Milano, Bocconi University and the Metabolic Bone Diseases and Osteoporosis Section of the IRCCS Humanitas Research Hospital. Approval for conducting all the phases of the study was obtained from the Ethics Committees of Politecnico di Milano (approval number 39/2021), Bocconi University (approval number 365/2021) and Humanitas Research Hospital (approval number 717/21). We obtained written informed consent from all patients prior to participating in every phase of the study. Figure 1 illustrates the four phases of the study contributing, on the one hand, to the development of the App, including its feasibility testing and, on the other hand, to the testing of its usability. The figure also illustrates how the participation and feedback of patients and Humanitas Research Hospital bone specialists were collected and incorporated in the development of the App and its usability testing.

Figure 1.

Phases of the study divided into the development and usability testing of the app.

The inclusion criteria applied to recruiting patients with osteoporosis to the different phases of the study are listed in Table 1.

Table 1.

Inclusion criteria for patients with osteoporosis recruited to the study.

Inclusion criteria	App development(Phase I, II, III)	Usability testing (Phase IV)
1. Diagnosis of osteoporosis	Yes	Yes
2. Smartphone with Android operative system and Internet access	Yes	Yes
3. Treatment for osteoporosis with oral bisphosphonates	No	Yes
4. Patients in treatment for osteoporosis at the IRCCS Humanitas Research Hospital	No	Yes

Phase I: focus group with patients with osteoporosis and joint interview with bone specialists

In phase I, six patients with osteoporosis were involved in a focus group to discuss their needs and expectations with respect to how to best manage their chronic condition and to increase adherence to drug therapy for osteoporosis with the help of an app. The focus group was held in person at Bocconi University on November 29, 2021, and lasted 2 hours. No patient dropped out from Phase I, and only one focus group took place. It was moderated by researchers of Bocconi University and Politecnico di Milano and conducted according to the best practices for managing focus groups in participatory design studies.²⁶ Amelia Compagni conducted the focus group. Amelia Compagni is a female researcher with a PhD who is Associate Professor at Bocconi University; she is a teacher of qualitative research methods at the PhD level, conducted numerous focus groups in a variety of projects in the healthcare field and received internal training on interviewing and running focus groups at Bocconi University. The discussion was not audio-taped but, during the 2 hours, the interviewer (Amelia Compagni) wrote on a whiteboard the main themes and concepts emerging from the participants, using as much as possible participants’ words, while the rest of the facilitators (Benedetta Pongiglione, Alessandra Angelucci, Andrea Aliverti) took notes. No one was present at the focus group besides participants and researchers. At the end of the focus group, the whiteboard and each facilitator's notes were photographed and the material jointly analyzed by the researchers of Bocconi University and Politecnico di Milano leading to compile a list of the main themes and of suggestions with respect to the app emerging from patients.

No relationship was established prior to study commencement. The patients were recruited among the members of the Federation of all Italian Patient Associations on Osteoporosis (FEDIOS). The recruiting was conducted by FEDIOS and occurred based on the criteria listed in Table 1 and based on convenience, that is, based on the willingness of the patients to participate in the focus group. The participants were approached by means of an email by FEDIOS with a very brief presentation of the study and the inclusion criteria reported in Table 1. The participants knew nothing about the researcher apart from Amelia Compagni's involvement in the research project. The characteristics reported about the interviewer were her interest in using the ideas of the focus group to support the development of an app to be used by patients with osteoporosis.

The methodological orientation which underpinned the study is in reference to participatory co-design studies. The focus group was organized in two parts. In the first part, the six participants were asked to identify moments in which they felt the care and management of their osteoporosis was optimal and their adherence to drug therapy was high. The second part of the focus group centered on which functions an app would need to display to best support patients in the management of their osteoporosis and in adhering to therapies. A goal was to guarantee that all the six participants contributed equally to the discussion and that some degree of thematic saturation could be reached.²⁷ For this reason, the focus group was conducted in ways that in each part participants had to progressively add their perspective until there was nothing more or different they could add. Once this happened, the main facilitator prompted discussion with an additional question.²⁸ The discussion was very lively and all the six participants put forward suggestions about the app. At the end of the focus group, three of the participants also volunteered to provide further feedback during the development of the app and to test it on their smartphones during the feasibility testing.

Notes from the focus group were not coded, therefore no coding tree is available. Rather, notes taken by the various researchers during the focus group and the photo of the whiteboard on which Amelia Compagni wrote were compared and discussed until a list of themes and app functions was compiled. Themes were derived from data. No software was used to manage the data.

In a separate meeting, we collected suggestions from the two bone specialists (Flaminia Carrone, Gherardo Mazziotti) who are in charge of the Humanitas Research Hospital Metabolic Bone Diseases and Osteoporosis Section through a joint interview.²⁸ The joint interview was semi-structured, was held on November 10, 2021, and lasted 60 min. It was facilitated by two researchers from Bocconi University (Amelia Compagni, Benedetta Pongiglione), while three researchers from Politecnico di Milano (Alessandra Angelucci, Andrea Aliverti, Maria Laura Costantino) who took notes during the discussion. All facilitators were part of the research team. During the joint interview, specialists were first prompted by the interviewer (Amelia Compagni) with questions about what they thought were the most important factors affecting adherence to medication by patients with osteoporosis, with specific reference to oral bisphosphonates, and were left free to elaborate based on their experience and on what was described in the literature. Next, questions about the features that an app should display to increase the adherence to therapy of patients with osteoporosis were asked to the two bone specialists. Notes taken by the interviewer and the rest of the research team during the joint interview were then compared and discussed, and a summary of the main suggestions emerging from the bone specialists was compiled.

Phase II: app fast-prototyping

In Phase II, an app was developed by the Politecnico di Milano researchers after combining the inputs received in the two focus groups and selecting those common suggestions that could be more reasonably implemented. This phase lasted till March 2022. The costs of the entire process can be estimated in approximately €35.000 mainly in time spent in developing the app prototype, buying a tester smartphone, testing the app with the help of patients, and providing a sort of basic “customer service” able to correct problems or adjust the app to patients’ emerging needs (e.g. reminders).²⁹

Android Studio software was used to create the app in the Android environment. Android Studio is the official integrated development environment (IDE) for Google's Android operating system, built on JetBrains’ IntelliJ IDEA software and designed specifically for Android development. The app was built in the Java programming language and based on Google Firebase as a backend service.

Firebase is a platform, powered by Google since 2014, which can drive and support apps’ backend, like data storage, user authentication and static hosting. Among the various functions provided by Firebase, two were used to develop the present app: Authentication and Real-time Database. Authentication provides services and application programming interfaces (APIs) to authenticate users. It supports authentication through password, phone number or other identity providers like Google or Facebook. In the case of the present project, login was possible via email and password. Real-time Database is a cloud-hosted database that allows storing and synchronizing between users in real time. This function was used to store patients’ information, such as frequency of drug therapy and the beginning of the reminder cycle. Other stored information was the requests sent by the patient to the bone specialist and the time at which the patient declared to have taken the prescribed medication.

Phase III: feasibility testing and further refinement of the app

Between April and May 2022 (Phase III, feasibility testing), the app was tested for 13–25 days and evaluated in terms of functionality by eight patients with osteoporosis recruited, as in phase I, through FEDIOS. The criteria for patient recruiting are listed in Table 1. Three of the recruited patients had participated in the focus group in Phase I. None of the recruited patients assumed oral bisphosphonates for their osteoporosis—that is, the drug therapy whose adherence the app was supposed to improve. As such, the reminder function (see later for details on the app functionalities) was tested only in terms of its technical functioning, that is, a fictitious reminder was set up and then patients were asked to confirm whether they had received it. The eight participants in the feasibility testing downloaded the app with the support of an engineer of the Politecnico di Milano development team, either in person or via a videotelephony software program, for example, Microsoft Zoom. During this phase, the patients had the opportunity to contact the Politecnico di Milano development team by email or phone and received support if they encountered any technical problems. After testing the app, all the eight patients were contacted separately by the Politecnico di Milano researchers for a face-to-face meeting in which they were administered the mHealth App Usability Questionnaire (MAUQ),³⁰ a validated questionnaire to assess their perception of app usability and functionality, and could provide some qualitative feedback about their experience and suggestions for improving the app. We obtained permission to use the questionnaire from the first author of the paper introducing the MAUQ.³⁰ The questionnaire included 20 questions with answers based on a 5-point Likert scale—from total disagreement to total agreement. Table S1 in Multimedia Appendix 1 reports the questions of the questionnaire divided into three subscales, that is, ease of use and satisfaction, system information arrangement and usefulness. A question from the original MAUQ (i.e. “The navigation was consistent when moving between screens”) was eliminated from the questionnaire due to the limited digital literacy of the test population. Besides the MAUQ, the main demographic characteristics (i.e. age and education) of the patients were collected as well as a self-assessment of their own digital skills. Patients were asked if they were able to surf the Internet or send an email and how they assessed their digital skills on a scale between excellent and very bad.

Based on the inputs of the respondents, the app prototype was further refined to reach the final version in June 2022.

Phase IV: usability testing of the app

To test the app for usability in a real-world setting, between June and September 2022, nine patients with osteoporosis were recruited by convenience sampling (i.e. based on their willingness to participate in the study) by the two Humanitas Research Hospital bone specialists among the patients referring for the first time to the Metabolic Bone Diseases and Osteoporosis Section of the hospital. All the recruited patients were diagnosed with osteoporosis for the first time and naive to treatment with oral bisphosphonates, which they were instructed to start taking once a week. As indicated in Table 1, an additional inclusion criterion was that they were Android user. None of the nine patients had been involved in the previous phases of the study.

The app was installed on the smartphone of each of the recruited patients and explained by one of the Humanitas Research Hospital bone specialists during the first visit. On this occasion, the bone specialist helped the patient set the reminder for the uptake of the oral bisphosphonates, by choosing the appropriate day of the week and time of the day. The usability testing lasted for 6 months and the Humanitas Research Hospital bone specialists were the primary contact for the patients with respect to the app throughout this phase. Patients were also provided with the email address of the technical support (i.e. the Politecnico di Milano app development team) but they never used it in case of technical issues with the app. Six months after their recruitment, patients were asked, after a follow-up visit with the Humanitas Research Hospital bone specialists, to fill in the MAUQ questionnaire, as used in the previous phase. In addition, the demographics of the patients were collected as well as their self-assessment of digital skills. Finally, an open question about possible points of improvement of the app was included at the end of the questionnaire. The questionnaire was administered by email to the enrolled patients through the survey platform Qualtrics.

Statistical analysis

Averages of the answers to the MAUQ questionnaire were calculated, respectively, for the feasibility testing group (eight patients) and for the usability testing group (nine patients), by item and by domain—that is, “ease of use and satisfaction” (MAUQ_E), “system information arrangement” (MAUQ_S) and “usefulness” (MAUQ_U). Averages by domain for the two testing groups were then compared using the Student's t-test, and a resulting p-value ≤ .05 was considered as a statistically significant difference between the feasibility and the usability testing groups.

Results

Phase I: app functions needed to promote medication adherence as emerging from patients with osteoporosis and bone specialists

Three main themes emerged from the focus group with the six patients with osteoporosis with respect to possible ways to promote adherence to medication for osteoporosis. The first emerging theme referred to the need for clear and concise answers to key questions on osteoporosis and its treatment. The participants in the focus group underlined how an app able to provide such information would prevent them becoming overwhelmed and confused while surfing the Internet for information, especially if connected to drug treatments and their potential side effects.

The second aspect emerging from the discussion with patients was the importance of frequent feedback and encouragement by the bone specialist in managing osteoporosis. Given that osteoporosis is for the most part asymptomatic, some encouragement in continuing with the therapy was deemed necessary by patients and relevant to maintain adequate adherence. For the same reason, ease of contact with the bone specialist to resolve doubts about the drug therapy or in case of side effects was considered important. Third, a high priority for patients was that of receiving advice on lifestyle issues, such as the appropriate level of physical activity and diet for their condition. In the second part of the focus group, focused on the app and its functions, patients underlined how an app should respond to all the three needs made explicit above. With respect to the last one (i.e. advice on lifestyle issues), the six patients agreed that a good compromise could be that of having at least a way to count their daily steps and compared them to an activity target recognized by the medical community.

As the focus group was not audio- nor video-recorded, no quotations are available, however, there was consistency between the data and the findings.

The joint interview with the two bone specialists allowed to identify three main functions which an app should fulfill to be able to improve adherence to drug therapy (i.e. oral bisphosphonates) in patients with osteoporosis. The first function was that of providing some valid and evidence-based information about osteoporosis and about its main drug therapies. The second function indicated by the bone specialists was that of allowing patients to contact the specialist outside of pre-set follow-up appointments to talk about problems with the drug therapy, mainly in terms of side effects. The specialists indicated that the first 6 months of treatment with oral bisphosphonates are those when patients are at higher risk of therapy discontinuation or poor compliance. Having a chance to contact the specialist when problems occur may allow patients to switch to an alternative therapy earlier without interrupting treatment. Finally, the specialists indicated the need for an app that could provide patients with reminders to take the drug therapy.

Table 2 summarizes and compares the functions of the app emerging from the focus group and needed to promote adherence to medication in the perspective of patients with osteoporosis and bone specialists, respectively.

Table 2.

App functions emerging from the focus group.

Patients with osteoporosis	Bone specialists
The app should:
• provide information about osteoporosis and its treatments	• provide information about osteoporosis and its treatments
• allow easy contact with bone specialist • provide/contain advice about lifestyle (diet and physical activity) • allow the bone specialist to encourage patient to continue with therapy	• allow easy contact with bone specialist • provide reminders to patients for taking drugs

By combining the inputs of the six patients with osteoporosis and the two bone specialists, we decided to incorporate four main functions into the app, as described below.

Phase II: functions included in the app prototype

The first app prototype contained four main sections, which performed the different functions described above. Figure 2(a)–(d) displays the respective sections of the app translated into English.

Figure 2.

Selected screenshots of the developed app: (a) “contact” section, to request an extra consultation with the bone specialist; (b) “FAQs” section; (c) “adherence” section part 1, to set date, time and frequency of reminders, and (d) “adherence” section part 2, where patients declared if they had taken the drug.

The first section, called “Contact,” gave the patient the option to send a request for an extra consultation with the bone specialist outside of fixed follow-up appointments (Figure 2(a)). When asking for additional contact with the specialist, patients could specify the main reason for the request, as the following: intolerance to drug therapy; information on drug therapy; relevant events (e.g. falls); other. If “other” was selected, the patient could add a text with further details. The request was sent to the Humanitas Research Hospital bone specialists in the form of an email.

The second section, called “Frequently asked questions” (FAQs, Figure 2(b)) contained 10 such questions about osteoporosis with their respective answers. For this purpose, specialists drafted a list of 10 FAQs and corresponding answers and the patients involved in the focus group approved them or asked for specific revisions. This section was only meant for consultations and had no interactive features.

The third section, “Adherence,” consisted of two parts. One part was made to set up the reminder for taking the drug therapy (Figure 2(c)) and could be accessed through a password known only to the bone specialist, so that patients alone could not modify the reminder without the specialist's approval. In this part, it was possible to insert the commercial name of the drug therapy, select the frequency of the reminder and the date and hour at which the reminders started. The frequency options were “once a week,” “once a month,” and “twice a month” (in the latter case, the medication is taken on two consecutive days of the same month). The set time was a result of a dialogue between the patient and the specialist. The hour of the day and the day of the week were chosen so that they would suit the patient's habits and routines while maintaining a clinically meaningful time of the day and frequency, that is, 30–45 minutes before breakfast. Once the frequency was set, the patient received a reminder at the chosen time in the form of a smartphone notification. The date and hour are selected on the phone, so in case the time zone of the phone changes, the reminder is received at the intended hour in the new time zone.

The other part of the “Adherence” section (Figure 2(d)) could be accessed via the main menu of the app or by clicking the smartphone notification. When the medication was taken, the patient could click the “send” button. If the medication was taken in a two-hour window around the optimal time, the following positive message appeared on the screen: “Congratulations! You took the drug at the right time. This way, you have reduced your fracture risk.” Otherwise, the patient received a warning message: “Attention! Remember that it is important to take the medication at the right time. When you don’t take the drug correctly, you increase your risk of fracture.” The data related to drug taking were recorded in Google Firebase and could be used to evaluate the adherence to the drug therapy.

Finally, the fourth section was called “Activity” and comprised of a step-counting function (the screenshot is not shown in Figure 2 because it was not included in the final version of the app). The number of daily steps was shown together with a target number (10,000 steps per day, which is an activity threshold widely recognized by the medical community³¹) and a progress bar visually showed how many steps were left to reach the target. In addition, for each day of the week, it was indicated, with a crown-shaped icon above the name of the weekday, whether the target had been reached. If the target had not been reached, no additional icon would appear.

Phase III: feasibility testing and assessment of the app by patients with osteoporosis

Table 3 summarizes the main characteristics in terms of demographics and digital skills of the patients with osteoporosis recruited in the feasibility and usability testing. It also displays the main differences between the two groups based on their enrollment modality, the main point of reference during testing and the length of testing.

Table 3.

Characteristics of the patients enrolled in the feasibility and usability testing.

Demographics of enrolled patients	Feasibility testing	Usability testing
Number	8	9
Sex (% woman) Age (years, average ± sd)	100 66 ± 9.5	100 57 ± 9.5
Education: % with at least a university degree	50	10
Familiarity with Internet/email: % capable of using Internet or email	100	100
Digital skills (self-reported): % with good or excellent digital skills	37.5	60
Enrolment among	Members of osteoporosis patient association	Newly diagnosed patients with osteoporosis
Main point of reference during testing	App development team	Bone specialists
Duration of testing	13–25 days	6 months

As shown in Table 3, the eight patients involved in the feasibility testing were all women, with a mean age of 66 ± 9.5 years and four of them (50%) had at least a university degree. All displayed familiarity with surfing the Internet and using emails while only 37,5% of them assessed their digital skills as good or excellent.

The assessment by the eight patients of the usability and functionality of the app gathered through the MAUQ questionnaire was overall positive. Figure 3 displays the average answers to the 20 questions in the feasibility testing (pink continuous line). The overall average value of the answers was 3.89 and varied between a minimum of 3.33 (Q9: “Whenever I made a mistake using the app, I could recover easily and quickly”) to a maximum of 4.33 (Q19: “I felt confident that any information I sent to my provider using the app would be received”).

Figure 3.

Average values of the answers to the MAUQ questionnaire in the feasibility testing (pink continuous line) and in the usability testing groups (blue dotted line). Ideal results corresponding to 5 are indicated with a violet dotted line.

Table 4 reports, for the feasibility and usability testing, the results of the MAUQ questionnaire subdivided into the three dimensions (MAUQ_E, MAUQ_S and MAUQ_U) and the difference between the two testing groups.

Table 4.

Average values of answers to the MAUQ questionnaire by dimension, difference between the feasibility and usability testing phases and results of the Student's t-test.

MAUQ Dimension	Feasibility testing	Usability testing	Difference	p-Value
Ease of use and satisfaction (MAUQ_E)	4.01	3.08	0.93	p = .008
System information arrangement (MAUQ_S)	3.64	2.76	0.88	p = .03
Usefulness (MAUQ_U)	3.90	3.16	0.74	p = .03

Statistical significance for p ≤ .05

Besides filling in the MAUQ questionnaire, at the end of the feasibility testing the eight patients also provided some qualitative feedback about possible improvements to the app. The qualitative input of patients suggested the removal of the step-counting function for two reasons. One reason was technical, that is, not all the patients in the population had smartphones with adequate hardware and firmware to extract step counts. The other reason was substantial, as some patients reported that they did not find it useful to have this function because they were advised by their doctors to perform other kinds of physical activity than walking. For these reasons, this function was removed in the final version of the app to be tested for usability in the real world.

Phase IV: usability testing and assessment of the app by patients with osteoporosis

As shown in Table 3, the nine patients involved in the usability testing were all women, with a mean age of 57 ± 9.4 years, and while only one patient had a university degree, 60% of them assessed their digital skills as good or excellent. The usability testing group was sensibly younger, less educated, and more digitally skilled in comparison to the feasibility testing group.

The only data collected through the app were the number of requests sent to the specialist and when the patient declared to have taken the medication. The expected occurrence of taking the medication was 24.9 times per patient, corresponding to 6 months of testing, but the occurrence declared through the app was on average 6.4 times. While one patient confirmed through the app to have taken the medication for 20 weeks out of 21, other patients confirmed this only once in the 6 months of testing.

Responses to the MAUQ questionnaire after usability testing are illustrated in Figure 3 (blue dotted line). The overall average value of the answers was 3.03 and varied between a minimum of 2.33 (Q13: “This app has all the functions and capabilities I expect it to have”) and a maximum of 3.56 (Q19: “I felt confident that any information I sent to my provider using the app would be received”). Despite being still reasonably positive, the assessment by the usability testing group was systematically worse than that of the feasibility testing group across all questionnaire items.

The table indicates that the highest difference (0.93) between the two sets of answers referred to the ease of use and satisfaction while the lowest average score referred to the system information arrangement dimension and was given by patients in the usability testing group (2.76). The difference between MAUQ_E, MAUQ_S and MAUQ_U results was found to be statistically significant by the Student's t-test (p = .008, p = .03 and p = .03, respectively).

During the 6 months of usability testing of the app, despite the possibility of receiving technical support from the Politecnico di Milano app development team, none of the patients contacted the technical support. The patients' feedback about the app was given only to the bone specialists. Bone specialists reported that some patients encountered problems in changing the date or time of the reminder on their own. Others appeared to find it difficult to use the “Contact” function to send an email to the bone specialist. Given that complaints were voiced to the bone specialist, it was then the bone specialist who contacted the Politecnico di Milano team to either correct the problem or ask for clarification. This made the entire process quite laborious and engendered some frustration on the part of patients. The bone specialists also found these drawbacks frustrating, as they were forced to deal with issues out of their medical expertise and scope of practice, and in which they felt inadequate.

Discussion

Overall, the development of the app and its feasibility and usability testing lasted 1 year and 3 months.

The journey began with a focus group with patients with osteoporosis and with a joint interview with bone specialists, that allowed the identification, in a participatory way, of the main functions that an app should display to potentially help patients with osteoporosis to adhere to their drug therapy. A participatory design process has been highlighted in the literature²³ as a very important determinant of the success of mHealth interventions in the management of osteoporosis. This method was employed in the development of mHealth applications for different medical conditions such as multiple sclerosis³² and COPD,^33,34 demonstrating a notable degree of acceptance of the resulting applications among both patients and healthcare professionals.

The capacity of mHealth interventions to actually improve adherence to medical treatment and self-management has been recently documented in the literature for different chronic diseases such as multiple sclerosis, pulmonary tuberculosis, and diabetes.³⁵ Studies show that a noteworthy enhancement in medication adherence can be achieved through app functions such as delivery of medical guidance, offer of educational resources about the chronic condition, monitoring of medication usage, and enabling of remote or more frequent specialist consultations.

In our case, the functions of the app emerging from the participatory process with patients and bone specialists were very much in line with what was described above. For instance, the “Frequently Asked Questions” section had the role of providing accessible information about osteoporosis to the app users. Previous studies have shown that access to information on osteoporosis is used by patients as a tool to improve dialogue with their specialists.^36,37 Furthermore, this section conveyed a sense of trustworthiness and expertise, two crucial attributes that a mHealth app should possess to be deemed effective, according to principles of persuasive system design (PSD).³² Another important function of the app is the opportunity for patients to set their medication times at the beginning of the test together with the bone specialist. The times were chosen according to the individual patients’ habits and routines, while always maintaining a clinically meaningful self-administration time. This allowed to have personalized reminders consistent with the framework of person-centered design but at the same time to be compliant with the specialist's recommendations. In fact, the medication should be taken 30–45 min before breakfast for it to be effective, but the time was chosen to accommodate preferred wake up and breakfast times of the individual patient. Personalized reminders and consequent encouragement through positive messages represented additional elements in line with the PSD framework. This app function offered valuable feedback and support to patients in their journey toward achieving the target behavior.

Finally, the “contact” function of the app granted patients the possibility to contact specialists outside of fixed appointments and request an additional consultation, making potentially easier the discussion with the specialists of doubts about the therapy or side-effects, as shown for other chronic conditions such as COPD and heart failure for which medication adherence is critical to attain good health outcomes.³⁶

In our journey, the feasibility testing, first, and then the usability testing offered patients a chance to experiment with the app and critically assess its strengths and limitations and, as a result, this has permitted to revise the app. In particular, the usability testing allowed to check strengths and limitations of the app for a longer time and in the real context of the daily management of osteoporosis. The two testing phases returned similar results as well as interesting differences in the assessment by patients of the app and its usability. On the one hand, the perception that the app could be used effectively to manage their osteoporosis and improve adherence to therapy were high for both patient groups. In fact, the average score of Q16 in the MAUQ questionnaire (“The app helped me manage my health effectively”) was very similar in the two phases. This may be suggestive that the functions identified in the focus group were indeed considered relevant by patients for the purpose of managing well osteoporosis and its treatment.

On the other hand, all answers to the MAUQ questionnaire obtained higher scores in the feasibility than in the usability testing. Three main differences may help explain this result: the main support figure in the two phases, the different duration of testing, and the different level of digital skills between the two groups. First, during the feasibility testing, for any problem with the app, patients could directly contact the engineers of the app development team, who were available for direct questions and clarifications. In the usability testing, instead, the contact with the technical support was mediated by the bone specialists. Although the specialists received training on the app, they were not part of the technical development team and were themselves “naïve users” of the app, which may have led to less clarity and promptness in addressing patients’ technical problems, and, as such, a worse assessment by this patient group of the app. A good “customer service,” early technical support and resolving of issues may, therefore, be critical in encouraging the use of mHealth applications developed for the management of chronic conditions and for higher levels of satisfaction by app users. This has also been observed in other medical fields. A study about an app for chronic traumatic brain injury,³⁸ for instance, compared a group of patients using the app before and during the COVID-19 pandemic, and the use of the app was higher in the former group. The authors suggested that lower usage could be due to less hands-on training and lack of ongoing technical support experienced during the pandemic.

Another explanation for the lower assessment of the app by the usability testing group is that these patients used it for a much longer period. In the literature, a well-known phenomenon is that of technology fatigue, one of the factors that hampers technology acceptance.³⁹ Patients may feel overwhelmed over time by the number of devices needed to navigate the day⁴⁰ and eventually lose interest in the technological tool,⁴¹ leading to a worse perception of its usability or utility.

Finally, the usability patient group was sensibly younger and more digitally skilled than the feasibility user group. These patients might have been not only more familiar with the use of applications for health purposes but also more demanding about the functions that the app should display. This is confirmed by the fact that the lowest score in the MAUQ questionnaire for this group (Q13) was related to the app having “all the functions and capabilities I expect it to have.”

Limitations

One limitation of the present study is the small sample of patients and bone specialists involved in the initial participatory phase. While the number of participants in the focus group is considered acceptable by the literature,⁴² and the focus group was facilitated in a way to collect the highest diversity of perspectives, it is possible that data or thematic saturation was not reached and that more focus groups or more participants would have led to the identification of more functions to be included in the app. The modularity of the app still grants the possibility to incorporate additional relevant functions in the future if needed.

The second limitation lies with the limited number of patients—overall 17—involved in the testing phases. However, even if limited, the number of patients enrolled in the feasibility and usability testing of this pilot study was sufficient to find statistically significant differences between the assessment of the app by the two groups. Moreover, in similar studies^34,43–45 a limited number of participants has been reported as sufficient to solve a considerable percentage of usability issues. In fact, prior research by Nielsen et al.⁴² has indicated that even with just five users, it is possible to detect approximately 85% of usability-related issues. For example, in the “My Osteoporosis Journey” study by Jakobsen et al.,³⁶ six women participated in the participatory design team.

Another limitation is the potential bias in the patients recruited to the study. Patients who gave their consent to participate in the feasibility testing were active members of a patient association, hence engaged with the treatment of their condition and part of a community, and more likely than those recruited in the usability testing to know about new technological and digital solutions applied to their chronic condition. In addition, three out of the eight patients involved in the feasibility study had participated also in the focus group, which could have increased their general acceptability and satisfaction with the app. Finally, the fact that the app was developed within the context of a research project and by researchers, may have led to additional challenges for final users that a commercial app would be less likely to display.

Conclusions

In this work, we developed an app to improve adherence of patients with osteoporosis to their drug therapies. The final version of the app provided information on osteoporosis, allowed patients to contact the bone specialist for an additional consultation, and generated a reminder for taking medications accompanied by feedback on adherence. The assessment of the app was positive but evaluations differed between the feasibility and usability testing, with the former displaying a significantly better assessment across all MAUQ items.

The innovativeness of the study lies with its participatory design and the two-stage testing of app usability. Use in the context of the daily management of osteoporosis led to positive yet worse evaluations of the app. This suggests that before an mHealth application can be considered satisfactory it might need repeated cycles of assessment and refinement conducted in collaboration with patients and healthcare professionals.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231218858 - Supplemental material for A participatory process to design an app to improve adherence to anti-osteoporotic therapies: A development and usability study

Supplemental material, sj-docx-1-dhj-10.1177_20552076231218858 for A participatory process to design an app to improve adherence to anti-osteoporotic therapies: A development and usability study by Alessandra Angelucci, Benedetta Pongiglione, Sara Bernasconi, Flaminia Carrone, Gherardo Mazziotti, Maria Laura Costantino, Andrea Aliverti and Amelia Compagni in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076231218858 - Supplemental material for A participatory process to design an app to improve adherence to anti-osteoporotic therapies: A development and usability study

Supplemental material, sj-docx-2-dhj-10.1177_20552076231218858 for A participatory process to design an app to improve adherence to anti-osteoporotic therapies: A development and usability study by Alessandra Angelucci, Benedetta Pongiglione, Sara Bernasconi, Flaminia Carrone, Gherardo Mazziotti, Maria Laura Costantino, Andrea Aliverti and Amelia Compagni in DIGITAL HEALTH

Footnotes

Acknowledgments

We would like to warmly thank the Italian Federation of all Italian Patient Associations on Osteoporosis (FEDIOS) and all the patients who voluntarily took part in the study.

Contributorship

AC, BP, AAn and AAl conducted the focus group. AAn designed the App. AAn, AAl, SB and MLC managed the App validation. AAn and SB administered and analyzed the questionnaire in the validation phase. AC and BP analyzed and interpreted the focus group discussion. BP analyzed the results of the MAUQ questionnaire. FC and GM recruited the patients to the usability testing and assisted them with the App installation and follow-up study. GM contacted FEDIOS. All authors interpreted the results of this paper. AAn, SB, BP and AC drafted the manuscript. All authors contributed to revisions of earlier drafts and read and approved the final manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Approval for conducting the study was obtained separately by the Ethical Committees of Bocconi University (approval number 365/2021), Humanitas Research Hospital (approval number 717/21) and Polytechnic of Milan (approval number 39/2021).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Fondazione Cariplo (grant number #2020-3978).

Guarantor: AC.

ORCID iD

Alessandra Angelucci

Supplemental material

Supplemental material for this article is available online.

References

Salari

Ghasemi

Mohammadi

, et al. The global prevalence of osteoporosis in the world: A comprehensive systematic review and meta-analysis. J Orthop Surg Res 2021; 16: 1–20.

Hampson

Stone

Lindsay

, et al. Diagnosis and management of osteoporosis during COVID-19: systematic review and practical guidance. Calcif Tissue Int 2021; 109: 351–362.

Epidemiology of osteoporosis and fragility fractures | International Osteoporosis Foundation, https://www.osteoporosis.foundation/facts-statistics/epidemiology-of-osteoporosis-and-fragility-fractures (accessed 9 July 2022).

Kanis

Norton

Harvey

, et al. SCOPE 2021: A new scorecard for osteoporosis in Europe. Arch Osteoporos 2021; 16: 1–82.

Johnell

Kanis

. An estimate of the worldwide prevalence and disability associated with osteoporotic fractures. Osteoporos Int 2006; 17: 1726–1733.

Palcu

Munce

Jaglal

, et al. Understanding patient experiences and challenges to osteoporosis care delivered virtually by telemedicine: A mixed methods study. Osteoporos Int 2020; 31: 351–361.

Conley

Adib

Adler

, et al. Secondary fracture prevention: consensus clinical recommendations from a multistakeholder coalition. J Bone Miner Res 2020; 35: 36–52.

Hernlund

Svedbom

Ivergård

, et al. Osteoporosis in the European union: medical management, epidemiology and economic burden. Arch Osteoporos 2013; 8: 1–115.

Mazziotti

Bilezikian

Canalis

, et al. New understanding and treatments for osteoporosis. Endocrine 2012; 41: 58–69.

10.

Adami

Isaia

Luisetto

, et al. Fracture incidence and characterization in patients on osteoporosis treatment: the ICARO study. J Bone Miner Res 2006; 21: 1565–1570.

11.

Adler

. Osteoporosis treatment: complexities and challenges. J Endocrinol Investig 2016; 39: 719–720.

12.

Solomon

Iversen

Avorn

, et al. Osteoporosis telephonic intervention to improve medication regimen adherence: A large, pragmatic, randomized controlled trial. Arch Intern Med 2012; 172: 477–483.

13.

Hiligsmann

Cornelissen

Vrijens

, et al. Determinants, consequences and potential solutions to poor adherence to anti-osteoporosis treatment: results of an expert group meeting organized by the European society for clinical and economic aspects of osteoporosis, osteoarthritis and musculoskeletal. Osteoporos Int 2019; 30: 2155–2165.

14.

Rothmann

Jakobsen

Jensen

, et al. Experiences of being diagnosed with osteoporosis: A meta-synthesis. Arch Osteoporos 2018; 13: 1–13.

15.

Keshishian

Boytsov

Burge

, et al. Examining the effect of medication adherence on risk of subsequent fracture among women with a fragility fracture in the US medicare population. J Manag Care Spec Pharm 2017; 23: 1178–1190.

16.

Shalev

Sharman Moser

Goldshtein

, et al. Adherence with bisphosphonates and long-term risk of hip fractures: A nested case-control study using real-world data. Ann Pharmacother 2017; 51: 757–767.

17.

Höer

Seidlitz

Gothe

, et al. Influence on persistence and adherence with oral bisphosphonates on fracture rates in osteoporosis. Patient Prefer Adherence 2009; 3: 25–30.

18.

Gadkari

McHorney

. Medication nonfulfillment rates and reasons: narrative systematic review. Curr Med Res Opin 2010; 26: 683–705.

19.

Fatoye

Smith

Gebrye

, et al. Real-world persistence and adherence with oral bisphosphonates for osteoporosis: A systematic review. BMJ Open 2019; 9: e027049.

20.

Pongiglione

Carrone

Angelucci

, et al. Patient characteristics associated with the acceptability of teleconsultation: A retrospective study of osteoporotic patients post-COVID-19. BMC Health Serv Res 2023; 23: 230.

21.

Slomian

Appelboom

Ethgen

, et al.

Can new information and communication technologies help in the management of osteoporosis?

Women’s Health 2014; 10: 229–232.

22.

Iribarren

Cato

Falzon

, et al. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS One 2017; 12: e0170581.

23.

Alhussein

Hadjileontiadis

. Digital health technologies for long-term self-management of osteoporosis: systematic review and meta-analysis. JMIR mHealth UHealth 2022; 10: e32557.

24.

Jakobsen

Wiil

Søndergaard

, et al. Using participatory design to design and develop mHealth for women with osteoporosis. Int J Integr Care 2016; 16: S23.

25.

Kirkscey

. Mhealth apps for older adults: A method for development and user experience design evaluation. J Tech Writ Commun 2021; 51: 199–217.

26.

Alsswey

Al-Samarraie

. Elderly users’ acceptance of mHealth user interface (UI) design-based culture: the moderator role of age. J Multimodal User Interfaces 2020; 14: 49–59.

27.

Hennink

Kaiser

Weber

. What influences saturation? Estimating sample sizes in focus group research. Qual Health Res 2019; 29: 1483–1496.

28.

Roulston

. Interviewing: A guide to theory and practice. Thousand Oaks, CA: SAGE Publications, Inc., 2021.

29.

Agarwal

LeFevre

Lee

, et al. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. Br Med J 2016; 352: i1174.

30.

Zhou

Bao

Setiawan

IMA

, et al. The mHealth app usability questionnaire (MAUQ): development and validation study. JMIR Mhealth Uhealth 2019; 7: e11500.

31.

Choi

BCK

Pak

AWP

Choi

JCL

. Daily step goal of 10,000 steps: A literature review. Clin Investig Med 2007; 30: E146–E151.

32.

Howard

Win

Guan

. Mobile apps used for people living with multiple sclerosis: A scoping review. Mult Scler Relat Disord 2023; 73: 104628.

33.

Naranjo-Rojas

Perula-de-Torres

LÁ

Molina-Recio

. Patients, caregivers, and healthcare professionals’ needs when designing the content of a mobile application for the clinical monitoring of patients with chronic obstructive pulmonary disease and home oxygen therapy: A user-centered design. Internet Interv 2022; 29: 100552.

34.

Naranjo-Rojas

Perula-de Torres

LÁ

Cruz-Mosquera

, et al. Usability of a mobile application for the clinical follow-up of patients with chronic obstructive pulmonary disease and home oxygen therapy. Int J Med Inform 2023; 175: 105089.

35.

Zhou

Wang

, et al. Effectiveness of Mobile medical apps in ensuring medication safety among patients with chronic diseases: systematic review and meta-analysis. JMIR mHealth UHealth 2022; 10: e39819.

36.

Ravn Jakobsen

Hermann

Søndergaard

, et al. Development of an mHealth application for women newly diagnosed with osteoporosis without preceding fractures: A participatory design approach. Int J Environ Res Public Health 2018; 15: 330.

37.

Ravn Jakobsen

Hermann

Søndergaard

, et al. Help at hand: women’s experiences of using a mobile health application upon diagnosis of asymptomatic osteoporosis. SAGE Open Med 2018; 6: 2050312118807617.

38.

Juengst

McShan

Conley

, et al. Feasibility and pilot testing of Mobile health apps to supplement 2 healthy lifestyle interventions in chronic traumatic brain injury. J Head Trauma Rehabil 2022; 37: 162–170.

39.

Venkatesh

Thong

JYL

. Unified theory of acceptance and use of technology: A synthesis and the road ahead. J Assoc Inf Syst 2016; 17: 328–376.

40.

Dash

Mohanty

. Technology readiness and the older citizen’s acceptance of m-health services in India. Digit Policy, Regul Gov.

41.

Thomas

Kumar

Onongaya

, et al. Explaining differences in the acceptability of 99DOTS, a cell phone-based strategy for monitoring adherence to tuberculosis medications: qualitative study of patients and health care providers. JMIR Mhealth Uhealth 2020; 8: e16634.

42.

Nielsen

Landauer

. A mathematical model of the finding of usability problems. In: Proceedings of the INTERACT ‘93 and CHI ‘93 Conference on Human Factors in Computing Systems, pp.206–213. New York, NY, USA: Association for Computing Machinery.

43.

Henshall

Davey

. Development of an app for lung cancer survivors (iEXHALE) to increase exercise activity and improve symptoms of fatigue, breathlessness and depression. Psychooncology 2020; 29: 139–147.

44.

Choemprayong

Charoenlap

Piromsopa

. Exploring usability issues of a smartphone-based physician-to-physician teleconsultation app in an orthopedic clinic: mixed methods study. JMIR Hum Factors 2021; 8: e31130.

45.

Storm

Fjellså

Skjærpe

, et al. Usability testing of a mobile health application for self-management of serious mental illness in a Norwegian community mental health setting. Int J Environ Res Public Health 2021; 18: 8667.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB