Sage Journals: Discover world-class research

Abstract

Mental health applications hold great promise as interventions for addressing common mental issues. Although many people with mental health issues use mobile app interventions, their adherence level remains low. Low engagement affects the effectiveness of mobile interventions. However, there is still a dearth of research to explain the reasons for low engagement. User experience and usability are two factors that determine the adoption and usage of apps. Analyzing user reviews of mobile apps for mental health issues reveals user experience and what features users liked and disliked in the apps and hence informs future app design and refinements. This research aims to analyze user reviews of publicly available mental health applications to uncover their strengths, weaknesses, and gaps, hence revealing why users are likely to cease using these applications. We mined reviews of 106 mental health apps retrieved from Apple’s App Store and Google Play and employed thematic analysis on 13,549 reviews. The review analysis shows that users placed more emphasis on the user interface and the user-friendliness of the app. Users also appreciated apps that present them with a variety of options, functionalities, and content that they can choose. Again, apps that offer adaptive functionalities that allow users to adapt some app features also received high ratings. In contrast, poor usability emerged as the most common reason for abandoning mental health apps. Other pitfalls include lack of a content variety, lack of personalization, lack of customer service and trust, and security and privacy issues.

Keywords

design recommendations human–computer interaction low engagement mental health apps user experiences

Introduction

Nowadays, mental health issues have become a major public health challenge. It can impact people’s mood and thoughts to the point of preventing them from achieving their daily tasks such as work and study¹ and may provoke some suicidal tendencies.² People who suffer from mental health issues often require assistance, including social support from family and friends, to cope with their conditions and avoid isolation.³ Consequently, many are using mental health digital applications as a source of support. There are more than 10,000 mental health and wellness apps available for download,⁴ providing a wide range of services such as information, meditation, and symptom tracking, online coaching, social support, and so on. Although numerous mobile health apps exist on the App Store and Google Play, few have been evaluated for effectiveness and usability. Moreover, given the proliferation of mental health applications, it is unlikely they have all been evaluated for effectiveness, usability, and safety.

Although some studies have shown that mental health applications could be effective at reducing mental health issues such as depression and anxiety, they have also highlighted high drop-out/attrition levels,^5–7 as a major issue hindering the adoption, use, and hence effectiveness of mental health applications. For example, 82 percent of the sample dropped out of the study by Roepke et al.,⁵ and 74 percent in the study by Arean et al.⁸ The reasons for these high attrition rates have not been well investigated, particularly from the user perspective. Thus, it is important to examine app efficiency, user experience, and satisfaction from the user’s point of view. This will help uncover the reasons for the high attrition and drop-out rates recorded by many mental health applications. Users with mental health issues usually need to use the apps for a long time to realize the full benefit and avoid relapse of mental health disorders.⁹ Research has not been able to show the long-time effectiveness of mental health apps because of the high attrition rates.¹⁰ Therefore, there is still a lack of evidence concerning app effectiveness over the long term.¹¹

The goal of this article is to analyze user reviews of publicly available mental health apps to understand user needs, experience, satisfaction, and expectations of mental health app. As a secondary objective, we aim to use the insights from the reviews to offer recommendations for designing effective mental health apps that are more usable and engaging and have lower attrition rates.

To achieve this, we mined and analyzed user reviews using thematic analysis to iteratively uncover important themes. Specifically, we mined user reviews of 106 mental health apps retrieved from Apple’s App Store and Google Play and employed thematic analysis on 13,549 reviews to identify key themes and gather more detailed insights. The results show that users highly appreciate user-friendly apps with a well-designed user interface (UI). Also, apps that offer a variety of options, functionalities, and customizable content are preferred by users and are more likely to keep them engaged for a longer time. In addition, apps that provide adaptive functionalities that allow users to adapt some app features are also highly appreciated by users. In contrast, poor usability emerged as the most common reason for app abandonment. A lack of option variety, personalization, customer service, trust, and security also led to dissatisfaction with the app.

This article contributes to advancing the state of the art by exploring the strengths and weaknesses of publicly available mental health applications and also sheds light on opportunities for future work in this area by offering some recommendations for developing mental health applications that meet user needs.

Literature review

In recent years, mental health issues have become a major source of concern in our society due to the increases in its occurrence and the devastating effects it has on the individual, their loved ones, and society as a whole. According to a survey by the World Health Organization—Mental Health, approximately 350 million individuals suffer from depression yearly.¹²

Digital health tools provide the opportunity to make health services accessible and more engaging. Specifically, smartphones and other handheld mobile devices are an attractive platform for delivering mental health interventions because of their ubiquitous nature coupled with their increased capacity to process and handle a large amount of data, deliver instant feedback, and share data with the healthcare providers and researchers.¹³

There are many applications designed to support mental health issues. Popular examples include apps that enable self-monitoring of personal mental health–related data such as mood¹⁴ and mental health games–based apps that help to manage anxiety and achieve healthy living.¹⁵ There are also many more apps available on the App Store and Google Play that are targeted at promoting mental health and tackling various mental health issues. These apps use various techniques including (1) tracking and monitoring to improve personal insight about causes and symptoms of mental illness, (2) providing techniques to mitigate and manage symptoms emerging as a result of a mental health issue such as breathing and meditation, and (3) providing a social community where users can ask and/or answer questions and find support.

Although mental health apps have proven effective at reducing mental health issues, there are still high levels of drop-outs/attrition. For example, Roepke et al.⁵ designed two versions of SuperBetter (SB) for depressed individuals, which can be accessed via smartphone or website (first version SB uses cognitive-behavioral therapy (CBT) and positive psychotherapy strategies and the second version of SB focuses on self-esteem and acceptance of the present). They conducted a study to compare the two versions of SuperBetter and found that the version based on CBT and positive psychology did not perform better than the version based on self-esteem and acceptance of the present. However, the two versions of SuperBetter had high attrition rates (approximately 21% of the sample completed the post-test for each of the versions). Moreover, Arean et al.⁶ conducted a study to compare three different self-guided mobile apps for depression (cognitive control app, problem-solving therapy app, and health tips app). For each intervention, they used reminders to notify the users about the new assessments. While results indicated that apps offering cognitive training and problem-solving made a stronger impact on depressed mood, 74 percent of registered participants dropped out from the study. Despite poor adherence in the above studies, the benefits seem to be positive in terms of reducing depressed mood.

There is minimal direct evidence regarding the potential reasons behind the low engagement and high attrition rate recorded by many mental health apps. According to Torous et al.,¹⁶ there are various possible reasons that explain the low engagement with mental health apps. First, poor usability can lead to low engagement. For example, Sarkar et al.¹⁷ conducted a usability evaluation across four common mood tracking apps. The study revealed that more than 50 percent of participants with a depressive disorder experienced some degree of difficulty and spent too much time entering their mood as data into the application and accessing this data. Similarly, Alqahtani and Orji⁸ identified usability issues in 106 mental health applications by conducting a thematic analysis of 1236 users’ reviews. They were able to classify usability issues in mental health apps into six categories: bugs and poor design of UI, data loss, battery and memory usage issue, lack of guidance and explanation, Internet connectivity issue, and missing important features. Second, privacy issues may make people hesitant to use a mental health app. According to Sunyaev et al.,¹⁸ among the 600 most common health apps, only 183 provided users with privacy policies. Third, some applications do not meet the needs of users. Recent research on self-management apps for people with bipolar disorder found that users were interested in a wide range of apps for self-management whereas available apps do not fit their needs and lack most of the desired functionality such as sleep-management, understanding of early warning signs and triggers, and features for developing wellness plans.¹⁹ Finally, a lack of trust may also contribute to low engagement. While a few mental health apps are based on evidence, a tremendous number (more than 10,000) of currently available apps⁴ are not based on scientific evidence and some may even be harmful. This might lead to lack of trust not only by users but also by healthcare providers.¹⁶ Some of the apps are unhelpful in emergency situations which may also contribute to low engagement. For example, Miner et al.²⁰ examined publicly available apps for current clinical evidence-based techniques for suicide prevention and found that none implemented best practices such as making comprehensive support visible and available within the app.

Therefore, understanding the opinions and experiences of users is critical if we aim to design mental health apps that will be used. As a result, this article contributes to advancing the state of the art by exploring the strengths and weaknesses of publicly available mental health applications and also offers some recommendations for developing mental health applications that will satisfy users’ needs. Specifically, from our qualitative analysis of user reviews, we reveal which design factors are important for users and which factors influence user adherence.

Methodology

User review analysis

Available marketplaces such as the App Store for Apple iOS platform and Google Play for Android platform allow users to download apps and evaluate them by providing a rating and writing comments. This publicly available data (user reviews) are regarded as a comprehensive evaluation of the app from the user’s own perspective. These reviews could include both positive and negative feedback and identify specific aspects or qualities that users like and dislike. Thus, user reviews are considered a rich source of information and a valuable source of insight. As a result, the approach of analyzing users’ reviews to gather insight has been used by many researchers. For example, Pagano and Maalej²¹ confirm that users’ reviews include valuable comments, user experience, bug reports, and feature requests which can help the developer respond to unmet needs from the users. Moreover, Stawarz et al.²² evaluated a medication reminder app by conducting a thematic analysis of users’ reviews of the app. From the reviews, they were able to understand how users use the app in the real world. Similarly, Nicholas et al.²³ used users’ reviews to investigate the users’ perspective regarding bipolar disorder apps. According to Thach,²⁴ this method allows the researcher to perform a quantitative summary of the main themes of the reviews and obtain rich insights into users’ perspective via qualitative analysis.

Selection of sample apps

The initial list of 437 apps was retrieved via a keyword search on the App Store and Google Play (254 apps from the App Store and 183 apps from Google Play). The following keywords were used to search the App Store and Google Play: “mental health,” “anxiety,” “depression,” “mood,” “emotions,” and “stress.” We also searched using various combinations of the keywords joined using the conjunctions “OR” and “AND.” Apps were examined by first reading the descriptions of the app and the app demos. Finally, we downloaded each app and examined the contents. For our analysis, we included apps whose main goal targeted mental health and apps that had more than five reviews (comments). Furthermore, apps were excluded if they fell into any of the following categories: not focused on mental health, had less than five reviews (we removed the apps that have less than five comments because these apps were less informative to review), was a duplicate app (appeared in both App Store and Google Play), or was not in English. After applying the selection criteria, 106 apps with a total of 13,549 reviews were included in this analysis (Figure 1).

Figure 1.

Process of selecting mental health apps.

Data collection and analysis

This study evaluates mental health apps by examining users’ viewpoint of the apps via a thematic analysis of user reviews. Users of mobile apps are able to download the apps, rate the apps, and write comments based on their experience and satisfaction. This publicly available data (user reviews) contains comprehensive evaluations which include positive, negative, or neutral feedback. Written reviews may also provide information regarding user satisfaction with specific app features, or they may express a desire for improvements. Thus, reviews are considered a rich source of information offering valuable insight. The App Store and Google Play do not provide a public API (Application Programming Interface) for automatically retrieving users’ reviews. Instead, we obtained the users’ reviews using a web-service called Heedzy, which gathers reviews of all iOS and Android apps.²⁵ A total of 13,549 reviews are included in the analysis based on the inclusion criteria. In addition to user reviews, the following information was extracted for each eligible app based on app classification with the MARS tool:²⁶ name, strategies/theoretical background, platform (iPhone, Android, or both), developer, category of the app, date of the last update, and price (free, fee-based, and free with in-app purchases (where developers provide a free version and a paid version if users want to upgrade or unlock additional features in the app)). More details about the apps included in the analysis can be found in Supplemental Appendix 1.

To analyze the users’ reviews, we conducted a thematic analysis²⁷ of the reviews. A thematic analysis was chosen because it enables analysis of a large data set in a systematic way that assists with the interpretation of patterns in the text while taking into account the context.²⁸ We used a qualitative thematic analysis, in line with previous research,^24,29 which confirmed that data derived from thematic analyses can be analyzed and reported qualitatively. Specifically, we followed the Braun and Clarke’s²⁷ six-phase framework for conducting a thematic analysis: (1) becoming familiar with the data, (2) generating initial codes, (3) searching for themes, (4) defining themes, (5) reviewing themes, and (6) writing up the results. We used open coding which means we did not have predefined codes. Thus, we expanded, developed, and modified the codes during the coding process as new themes emerged. Two researchers individually coded some sample reviews (approximately 2400 reviews) to identify a core set of codes. They immersed themselves in the data by reading and rereading users’ reviews and then generating the initial codes detected in the text. The researchers then compared initial codes, examined similarities and differences, and collaboratively decided on which codes to use. The percentage of agreement between the two researchers was about 85 percent overall. After choosing the codes, one researcher coded the remaining reviews and extended the set as necessary. For each new review, the researcher manually examined all text in the review and classified accordingly. If the review did not match with any existing code, the researcher created a new code. For any issue that arose during coding and any new code generated, a third researcher was involved to rectify it.

We used descriptive statistics to detail the number of reviewed apps, the frequency of each identified factor, the last app update, developer, price, and platform (Android or iPhone). We also employed independent-samples t-tests to compare the mean ratings of free and paid apps.

Results

Theoretical background of selected apps

Unlike Thuch²⁴ who focuses on apps using the CBT approach, we collected reviews of apps that use a combination of different approaches. We identified the strategies/theoretical background used in mental health apps and found that monitoring/tracking (feelings, thoughts, journals, or behaviors) are the most frequently employed strategy (with a total of 39 apps), followed by mindfulness/meditation methods (37 apps). Other strategies used in mental health apps were the relaxation (e.g. deep breathing, coping self-talk, distraction, game, music, natural sounds, and yoga) by 82 apps, CBT by 26 apps, information/education by 21 apps, advice/tips /skill training by 16 apps, feedback (graph and chart that show pattern of collected data) by 10 apps, and journaling by 11 apps. The less frequently used strategies in mental health apps include assessment (nine apps), strengths-based strategies which focus on building the strengths of a person (eight apps), social support (seven apps), professional support (five apps), gratitude (five apps), acceptance commitment therapy (three apps), hypnosis (two apps), and goal setting (one app), as detailed in Figure 2. Approximately 54 percent of the apps employed more than one strategy/theoretical background, making it difficult to highlight the effectiveness of individual strategies/theoretical backgrounds used in mental health apps.

Figure 2.

Strategies/theoretical background of mental health apps.

Descriptive of included apps

Approximately half (49/106, 46.23%) of the apps had been updated within the past year (2018). Moreover, the average price of 11 fee-based apps is US$5.26. Table 1 presents a summary of the price, developers (Figure 3), categories, and platforms of the apps.

Table 1.

A summary description of 106 mental health apps.

Price	Free (43 apps), fee-based (11 apps), Free with in-app purchases (52 apps)
Developer	Unknown (16 apps), Commercial (designed by business organizations) (72 apps), Government (8 apps), NGO (4 apps), University (6 apps) (see Figure 3).
Category	Health and fitness (73 apps), medical (17 apps), lifestyle (9 apps), education (2 apps), books (1 app), Game (1 app), Adventure (1 app), entertainment (1 app), social networking (1 app).
Platform	iPhone (25 apps), Android (29 apps), both (52 apps).

NGO: non-governmental organization.

Figure 3.

Developer of mental health apps.

User perceptions of the apps

App ratings quantitatively show users’ satisfaction with the apps. Regarding the users’ rating of all the considered apps, more than half of the apps were rated 4 stars and above (Figure 4). More specifically, a total of 77 of the apps were rated between 4 and 5 stars. Only four apps were rated below average (less than 3 stars; all four apps received ratings between 2 and 2.9 stars), and 18 apps received a rating between 3 and 3.9 stars. The remaining proportion of apps (7) do not have any rating.

Figure 4.

Rating of mental health apps.

The results of the independent-samples t-tests show that there was a significant difference in the ratings of free and paid apps (p = 0.034), where free apps (n = 95) received on average 4.023 stars, while paid apps (n = 11) received on average 3.173 stars. However, the significant difference in the numbers of paid and free apps may have affected the result.

With respect to the thematic analysis, we classified reviews into three categories: qualities that users liked, qualities that users requested, and weaknesses of mental health apps identified by users. These three categories help us explore how each component of the apps, and strategies employed, was designed to meet user’s needs and promote user adherence. Figure 5 presents the number of reviews contained in each category.

Figure 5.

Number of reviews of each category.

Qualities that users liked

Based on our analysis, users liked apps that are usable, offer a variety of features and options, are personalized, affordable, informative, credible, and secure. They also like apps that offer some form of social support, customer service, and emergency support. Users also preferred apps that allow them to have some control over the app features and functionalities (Figure 6). Below, we provide more detail of each of the characteristics that are valued by the users.

Figure 6.

Qualities that users liked.

We refer to reviewer comments using [Rid], where R = reviewer and id = a unique number used in identifying each review (reviews from users are included verbatim throughout the article, including spelling and grammatical mistakes).

Usability

Usability is an important aspect of user experience with mental health applications:

Ease of use: among the reviews, most users highlighted simplicity and ease of use as one the characteristics of the mental health apps that they really liked as shown by the sample review number 44, for example, “Simple and intuitive to use, well-structured and eases you into things” [R44].

Visually appealing UI: users liked apps that are well structured with a visually appealing UI, for example, “I like the color, well-designed interface and information is presented in a non-overwhelming way” [R35]. Moreover, users liked visually appealing games and charts. For example, “game colours and graphics are surprisingly therapeutic” [R6], “The colors and charts provide lots of useful feedback to monitor the ups and downs” [R202].

Clear guideline and instruction: users of mental health apps expressed a strong preference for apps with simple and clear guidelines and instructions regarding the use, irrespective of app appearance. For example, “Users of the app are guided step by step in using every aspect to support their emotional health” [R73]. Moreover, clear instructions of how to breathe properly and how to perform an activity were also appreciated by users. For example, “I love the visuals and the ability to guide you through breathing exercises” [R320].

Regular update: users emphasized the importance of updating the app more frequently to comply with new phone features, other software requirements, and to keep users engaged. For example, “They’ve also done a great job of keeping it up to date and working properly” [R20].

Other important features: some other important features related to usability—that were frequently highlighted in the users’ reviews—include the following:

➢ Backup data: users appreciated apps that provided backup functionality, “Nice that you can back it all up with Dropbox” [R23].

➢ Export function: users liked apps that allow them to export health reports for printing or sending to a physician by email, for example, “The export to PDF and email work well” [R88].

➢ Syncs function: users liked the ability to sync the app with other devices or other health kits, for example, “I love how it syncs to the watch” [R45].

➢ Hashtag function: users also liked the hashtag function that allows them to organize and search specific personal data easily, for example, “Tags allow the app to organize your notes to better analyze and show them back to you” [R60].

Variety of features and options

Users of publicly available mental health apps vary significantly in their characteristics, belonging to various demographic and cultural groups. What works for one group of users may not necessarily work for others. Based on the reviews, users of the mental health apps appreciated the variety of options, functionalities, and content provided by some mental health applications. For example, “it has variety of things you can do such as a journal, play interactive games, read articles and take quizzes. Definitely worth a try!, Lots of cool exercises” [R165]. Users expressed a preference for a variety of meditation functionalities: short and long meditations; guided and unguided meditations; different target audiences (e.g. advanced, novice, adult, and kids). For example, “There are many meditations and sounds to choose from” [R94].

Moreover, users frequently highlighted the importance of a variety of tracking functionalities provided by some apps such as tracking medicine, sleep, mood and other healthy habits as well as the ability to set different types of reminders throughout the day for different purposes. For example, “Let’s me track a lot of stuff. Tracks your moods, sleep, nutrition, etc.” [R55], and “I also like how you can set various mindfulness reminders throughout the day” [R309].

Users appreciated the diversity of other components available in the apps such as a variety of challenges, suggestions, media (picture, sound, music), activities, games, symptoms, and coping methods.

Personalization

Apps that provided users with the ability to customize the appearance of the app and its functionality to fit the users’ personal needs were well received. Users liked to customize the app theme, as well as add their own media (picture, voice, music, sound, and video). For instance, “I like to be able to customize the pictures in it to include smiling friends or happy pets” [R208].

In addition, users expressed a preference for apps with customizable reminders that can be tailored to fit individual daily routines, for example, “The reminders are nice too, especially since they are easily customizable” [R190]. They also highlighted the importance of providing an option to customize the length and rate of the breathing sessions, for example, “The breathing exercises are great because I can set the type and time which I see as a great feature” [R487].

Users also liked apps that tailor functionalities based on user data, such as providing suggestions based on behavior and health data or daily quests based on mood. Users liked customizing some aspects of the apps such as adding their own mood, desirable activities, coping methods, challenges, symptoms, and goals. For example, “I like the custom colors and being able to do my own labels” [R255].

Affordability

Although some users don’t mind paying a reasonable amount, especially if the app is well designed, they generally appreciate free apps that are also well designed. For example, “It’s free and well made” [R78]. In general, users are highly annoyed by paid apps that do not deliver on promises or that are not well designed.

Information

Users of mental health apps expressed a preference for apps that provide rich information and explanation of mental health issues and methods used to reduce it. For instance, “Extremely informative, explains anxiety from a scientific perspective in a way that is clear and comforting” [R378].

Credibility

Credibility is another aspect that users appreciated:

Evidence-based: users liked apps that are based on scientific evidence, for instance, “It works, and the science behind it is impressive” [R7].

No ads: users preferred apps that show no advertisements, for example, “best of all no ads” [R423].

Accurate: apps that give accurate tests and results were positively perceived, “The test gives accurate results. I like it” [R76].

Social support

Chatting with people who have the same issues makes the user feel normal and offers some social support. From our analysis, users expressed a preference for sharing their issues and feelings with people having the same issues, but they liked to remain anonymous while doing this. For example, “It’s a really good way to connect and feel connected to other people who have the same problem as you; even if you think you’re alone” [R73] and “having the community is perfect for me. I’m not good at talking about my feelings but anonymously posting positive things, strategies or strengths, or even just venting is so helpful for me” [R80].

Customer service

Good customer service that responds quickly to users’ concerns or issues is desirable for mental health apps. For instance, “the developer is incredibly communicative, such as when I reported a bug and he gave me some advice” [R65].

User control

Users liked to be able to control the app features and functionalities and be able to use them without too many constraints. People like to be able to save a quote, turn music off and on, and enter or edit mood or journal entries from the previous day in case they missed it. For example, “I like the ability to turn off the sounds in the breathing section” [R5].

Security

Mental health apps designed to gather user’s data need to be protected. Users appreciated mobile apps that enable them to protect their data and information with a passcode. For example, “It’s also secure so I don’t worry about the information . . .” [R352].

Regular updates

Mental health apps can reduce attrition rates by performing regular content updates. Users liked to see new meditations, activities, backgrounds, pictures, quotes, and dashboard features added frequently. For instance, “. . . new ones are added frequently. Of all the meditation apps I’ve tried, this is my favorite” [R635].

Emergency contact

Allowing users to provide emergency contact in the mental health app is an important feature that could help them with panic or suicide situations. For example, “thank you, thank you for including the suicide hotline link . . . so important!” [R49].

Qualities that users requested

Coupled with requesting more free content, some users requested improved quality. For example, some users requested improved usability, credibility, and security; the increased variety of options, personalization settings, and informative content; access to some form of social support; and emergency support. Users also requested control over app features and functionality (Figure 7). Although these characteristics are highlighted among the qualities of the apps that users liked in general, they are also requested of other apps that do not possess these qualities. This highlights the importance of these qualities. Figure 8 shows some qualities that appeared in both liked and requested categories. For example, a variety of features and options are highlighted among the qualities of the apps that users liked in apps that implemented them; however, they were also requested of other apps that lacked the features, hence the overlap.

Figure 7.

Qualities that users requested.

Figure 8.

Common features appeared in requested and liked qualities categories.

Usability

Users requested some improvement regarding the usability of mental health applications as follows:

UI: among the usability reviews, users requested improvements to non-professionally designed UIs, for example, “It would be awesome if there was a category to sort voice, voice with music, or voice with ambient noises, instead of having to preview each one” [R71].

App updates: users also requested regular updates for apps with outdated content and apps that are rarely updated, “Should be updated for iPhone 6 displays!” [R93].

Provide user guide: providing guidance and instruction on how to use the app, and how to play games were highlighted by users of apps with no clear user manual: “I wish it had more guided options” [R45].

Offline use: the possibility to use the apps offline was requested for apps that are heavily Internet-dependent. This would allow users to use the app when there is no Internet connection, “To that avail I wish you could use it at times when you don’t have an Internet connection” [R22].

Some other important functions related to usability were also requested:

➢ Landscape mode: “I wish there is landscape mode for writing in journal” [R64].

➢ Backup data: “I wish there was a mechanism to back up and restore data” [R33].

➢ Export data: “wish I could export my info. in graph form so I could share with my doctor more easily” [R75].

➢ Sync data: “there needs to be a way to sync the data from one device to another. The absence of that makes the app somewhat less useful” [R84].

➢ Hashtag and calendar: “I hope that a few features could be added: . . ., maybe a hashtag ability in the notes section, . . .” [R436]; “ability to create a calendar with all planned upcoming meditation . . .” [R543].

Variety of options

Providing a variety of options and functionality is important to keep users engaged. For example, users requested a variety of meditation types and narrative voices, “I would like to see more variations of the recording to switch between” [R26], and “wish there were more voice selections to choose from” [R79].

Users also expressed a desire for a variety of trackers to track different matrices such as mood, medicine, and sleep, “Would be awesome if you could track physical symptoms of depression/anxiety, too” [R30], and different type of data visualizations, “get more graphs of a similar type just with more variety” [R35].

Moreover, users would like more emoticons and emojis to choose from “more variety in the daily mood rating. I don’t feel like three mood levels is enough” [R59].

A variety of activities and tasks, games, and reminder settings were also requested. For example, “I wish there were more lessons and activities” [R29], and “add multiple reminders. I like to use the app in the morning, midday, and before bed” [R39].

Similarly, various forms of media such as music, sounds, and pictures were requested in the mental health apps, “it would be nice to have . . . more nature sounds or music options” [R85].

Users also asked for more variation for other features including breathing methods, coping strategies, and quotes. A variety of in-app challenges were requested. For example, a variety of questions or activities that users can do. Users also asked that apps include more symptoms or multiple symptoms to better reflect their situation.

Personalization

From the analysis, users requested personalized app functionalities that are tailored to each user and with different techniques relevant to individual situations. They prefer apps that can be customized to fit personal needs. For example, users request the ability to add their own mood, coping strategies, tasks, challenges, activities, music, sound, and pictures.

Moreover, users requested the ability to customize reminders, breathing rate and type, game character, and length of meditation. For example, “I hope adding a way to personali[z]e the app according to each user with different modes or methods that can really help them throughout their fight with depression” [R32].

Social support

Users requested some improvements to chatting features:

Make the communication anonymous: “It’s great for what it offers. I would have liked an anonymous community for support” [R437].

Chat monitoring and professional moderator: “Need some monitoring or censoring on the Social Cloud. chat boards there are no professionals to answer any questions just people trying to help people with their own stories/problems” [R428].

Other chat functions were also requested such as notification, reply, filters, and a block option.

Security features

Users would like apps to offer some security features. For instance, “I think it would be great to have a lock/pin feature to keep your personal thoughts private if others used your phone” [R511].

User control

From the analysis, users desire the ability to do a certain task without constraints (on time, for example) or limitations. For example, “Give us a list and we could pick one to do. Unlimited time to do the task” [R190]; “Option to do the task when users need and the ability to temporarily “pass” and come back to the quest later” [R117].

Credibility

Through their reviews, users requested information about app privacy policies. For example, “I wish there was more info on encryption. I’m syncing to iCloud, which is encrypted itself, but at least so far I haven’t seen anything that assures me my data is private and is only (and WILL only) ever viewable by me. It’s a small thing, but I’d like to know that” [R205].

Emergency contact

Users desired to have an emergency contact in case of emergency situations. For example, “. . . Should include an option to get help if user makes mention of suicidal or homicidal thoughts but fantastic otherwise” [R288] and “It would be nice though if it had a list of doctors or people to contact that could help me” [R278].

Information

More information about mental health issues was also desired by users of some applications. For example, “it would be great to include brief descriptions of the different cognitive distortions” [R495].

Weaknesses of mental health apps

The weaknesses of the mental health apps discussed by users were mostly related to usability issues, poor customer service, and unaffordability. Moreover, users complained about the lack of variety of options, personalization, control, trust, and security (Figure 9).

Figure 9.

Weaknesses of mental health apps.

Usability issues

Users complained about some usability issues including the following:

Bugs and poor UI design: most users discussed the issue of persistent bugs, such as crashing, functionality, and syntactic error; update issues; and download issues. For example, “The app opens up but once I try to click on anything it immediately closes down” [R300] and “After recent update, I’m not getting notifications” [R58]. Moreover, users complained about poor UI design such as layout/readability issues, and poor navigation. For example, “The layout is a little confusing and clunky and many buttons are small and hard to tap” [R83].

Data loss: data collected using a tracking application is extremely important for individuals with mental health issues, particularly those who spend a significant amount of time gathering data to improve their mental health and wellness. If the app develops an issue that causes users to lose this data, they may never use the app again. For example, “Have no clue what happened but I went back to look at old entries and all the photos were missing, even though the photos are still all on my phone. I spent hours putting in all of them and making notes and really enjoyed this app. Why are they all gone? I’m probably not going to use this app anymore since it’s removing all my important notes like this” [R66].

and, “love this app but it deleted all my data and it won’t save anything” [R32].

Battery and memory usage issue: this occurs when the app consumes a substantial amount of battery power and memory space. For example, “please update this to get rid of the memory leak bug, which slows down the keyboard input and freezes the app on large text input” [R155].

Lack of guidance and explanation: clear instructions and guidelines on how to use the app are crucial, especially for users with mental health issues that impair concentration and cause them to be easily frustrated. For example, “I have no idea how it worked . . . the only thing I could get to was my settings . . . and if I hit back it went to the same saying each time; no home screen no instructions; very disappointing and not user friendly” [R460].

Moreover, some app terms require clarification. For example, “explaining terms are missing” [R59] and “it should include brief descriptions of the different cognitive distortions” [R96]. Guidelines help reduce the learning curve and make the app easier to use.

Internet connectivity issues: a mental health app becomes less useful and accessible if it requires an Internet connection, since many users may not have Internet access and have to pay for data. For example, “can only use it when I have WIFI access, and I usually get panic attacks in public places where there is no WIFI. It does not load unless I have WIFI so I can’t really use it” [R72].

Missing important features: users complained about some missing features, such as back up, export, and sync capabilities. For example, “. . . too bad this app is not cloud-based” [R52], “There is no sync between device” [R30], and “unable to export report and print anything” [R76].

Untruthfulness

Users identified a number of issues that reduces app credibility and hence affects user trust:

Privacy issues:

➢ Lack of or weak privacy policy: “the privacy policy gives no solid reassurance that one’s data is well protected” [R42].

➢ Accessing information on the phone without permission: “This is unacceptable for an app that has access to location without permission, and access to sensitive health information” [R36].

➢ Some mental health apps ask for access to mobile data that is not related to the app use: “Records diary of intimate thoughts, demands permission to access my contacts, pictures and other personal information—no way” [R45].

➢ Asking users to create an account irritates some users: “Every app which isn’t a social media platform, that makes me create an account before I can use the app usually gets trashed” [R85].

➢ Asking for a lot of information during login: “This app asks for too much log in info and I don’t trust it” [R198].

Having ads on the app irritates users: “These pop-up ads are unacceptable in a paid app!!! I wish I hadn’t purchased multiple apps by them. Until these ads are removed or there is an option for the user to disable them, I suggest staying away” [R263].

Subscription issues: these include issues such as not allowing users to cancel a subscription when they want, paid features still locked after paying, and charging without notifying the users. For instance, “when I paid for the monthly subscription it didn’t unlock the short and long lessons” [R372] and “I’ve used the app once or twice in the past month . . . apparently, if you sign up for the free trial, they will bill you the full $100 annual cost without notifying you” [R524].

A developer without adequate background: if actual people behind the app’s content do not have an adequate background, it affects the app. For example, “found out this was created by people who know nothing of depression” [R511].

Unaffordability

Some reviewers complained about unaffordable apps. For example, “This app is so ridiculously overpriced. For more than Amazon Prime, you get access to all of the sleep stories and more meditations. What it does is good but this is seriously overpriced” [R411].

Lack of variety of options

Users complained about the lack of content variety, such as limited options for mood/feeling, meditations, and tracking. The user would like access to unlimited meditations and track options. For example, the app could have different options for tracking different metrics such as mood, sleep, medicine, and thought, “The problem is if you are doing the guided meditation everyday it is incredibly repetitive . . . If they can add more material and different types of meditation this app would be an easy 5 stars” [R483].

“There are seven primary emotions. Why restrict us to three”; “allows only negative responses” [R465].

“I stopped using it because it couldn’t easily include/accommodate things like physical exhaustion’s things which can be a symptom of mood or are neutral facts” [R629].

Poor customer service

Poor customer service is one of the issues users highlighted with some apps. Users get irritated when there is no response addressing app issues that have been highlighted. For instance, “no response to reporting this problem” [R307].

Lack of personalization

Some mental health applications are not personalized. Some reviewers complained about their inability to customize the app to fit their needs, “Don’t think the app can be customized enough for my personal needs” [R400].

Inability to customize the rate or length of breathing session, length of meditation, and the game characters was also highlighted by users: “There does not appear to be a way to adjust the breathing rate and it goes way too fast for me” [R206], “not have the ability to lengthen the meditation” [R505] and “lack of customization availability for the characters” [R333].

Lack of control

Users mentioned that they were not allowed to turn the sounds off or to go back to answer questions they missed which constrained them: “I am unable to turn off the sounds at the end of meditation” [R449].

Lack of security

Users complained about the lack of security in apps. Some apps that gather user data do not offer passcode protection to prevent external access. For example, “No option for PIN” [R525].

Discussion

Mobile apps have an important opportunity to enhance patients’ self-management of mental health issues. Analyzing user reviews is important for uncovering the strengths and weaknesses of mental health applications and opportunities for improvement. The reviews assist in understanding overall user experiences with the app and specific features users like and dislike. The results of our analysis show that in general, poor usability, lack of variety and personalization, credibility and security issues, and poor customer service emerged as the most frequently highlighted weaknesses of the mental health applications reviewed in this article. Interestingly, these features (usability, variety of features and options, personalization, credibility, informative, social support, user control, customer service, emergency contact, security features) are also the most frequently highlighted strengths (liked qualities) of applications that implemented them. They are also requested of applications that lack them. This shows how important they are for implementing successful mental health applications.

In general, most of the comments in the app reviews (both positive and negative) were related to usability. This finding is consistent with Sarkar et al.¹⁷ This shows that app usability is of utmost importance for designing an effective mental health application. Users are more likely to stop using an app due to usability issues that hinder their usage than any other technical and quality-related issues. This explains why there is a gap between the potential and reality of mental health applications with regard to achieving their intended objectives.

Mental health apps available on the App Store and Google Play lack usability evaluation. Therefore, the applications suffer from several design and interface issues, including bugs, poor UI, difficult-to-follow instructions and navigation, and a lack of orientation—all factors that, if addressed, would likely increase user engagement and reduce the high attrition rate.

Moreover, the reviews show that general mental health apps need to be designed to be suitable for a wide audience. Therefore, providing features that allow users to customize the app to suit their individual needs and preferences is a good way to improve user satisfaction, user experience, and hence reduce the high attrition rate. Users highlighted the need to customize most of the app features, such as types of meditation, games, and other activities to attract them and satisfy diverse user needs. For example, the app could allow users to set the length and rate of breathing exercises, and the type and duration of meditation as they deem suitable. This finding is similar to Price et al.³⁰ who found that personalizing some aspects of the app such as colors and backgrounds, and personalizing the assessment question content would improve app usability. Generally, allowing users to customize apps to suit them allows developers to meet more needs and preferences which will attract more users.

Overall lack of trust emerged as the second most frequently highlighted weaknesses of mental health apps, after poor usability. Users are highly concerned about disclosing their personal information when creating an account. Some mental health applications tend to request many personal details during account creation that, to many users, may seem unnecessary for achieving the purpose of the app. This raises some issues with respect to the trustworthiness of the app, especially for new users. In general, it seems that requesting too many personal details from users especially during account creation is undesirable as it makes users suspicious and leads to their abandoning the app. Hence, mental app developers should request only the basic information required to get an app set up and working during initial account creation. However, we acknowledge that a user’s personal details may be necessary to personalize the apps. Therefore, we suggest that designers could gradually collect more personal details as users get familiar with and gain some trust from using the app. However, the information collected and how it is stored, shared, and used should be transparent (made known to the user). Users should also be given opportunities to opt-in or approve of any personal information that is collected—full disclosure.

In addition, users highlighted the need for sync and backup features and discussed their concern with losing their data. It is impossible to implement these features without creating an account. As a result, it is important to clearly explain the main reason why users need to create an account and how that improves the app functionality and user experience before even requiring users to create the account. App designers can also offer a limited app version that users can use without creating an account, perhaps with minimal functionalities that do not need dedicated user information and allow the user to upgrade to a full version by creating an account and setting their profile when they feel the need. In general, apps should never access any user’s information that is not needed for the app, such as contacts, or access any information at all without explicit user’s permission.

Moreover, most mental health apps do not provide users with a privacy policy,³⁰ and it troubles the users and reduces app credibility. Therefore, mental health apps should provide clear information about their privacy policy in plain language. This reassures the users that their data are well protected and hence raises their trust in the app. Providing security features to protect data collected in the application (such as an app lock feature) is important for individuals with mental health issues due to the sensitivity of the information the app may contain. It would also increase app credibility and user trust.

Users with mental health issues liked to share their problems with people who have the same issue, so they feel less alone. However, they preferred to communicate anonymously and have moderators readily available to monitor for any issues, answer questions, and resolve any issues. This is probably difficult to achieve in free apps since maintaining a dedicated moderator requires money and developers may not be able to afford.

Similarly, price and customer service appeared to be important and could affect the use of apps. Free apps with valuable content would encourage app use. However, some free versions have little content and over time become repetitive, which could contribute to low engagement. However, overpriced apps or high prices to unlock more content could also discourage use. In addition, poor customer service negatively impacts the use of mental health apps. As users mentioned in their review, receiving no responses to reported issues or questions is extremely discouraging. A possible solution is to provide free apps and request some moderate renewable fees from people who would need some form of assistance/services such as moderators, or customer service.

Providing information that explains various mental health disorders, causes, symptoms, and the best way to overcome or manage them is extremely helpful and appreciated by users. Moreover, frequently adding new content to the mental health app (updating) and allowing unconstrained use of the app are other ways to decrease low-engagement with the app and retain users. Users easily get bored if apps do not update their content.

Regarding qualities that were appreciated when present and requested when absent, an option for emergency contact (even in offline mode) is very important for users suffering from a mental health disorder. Ability to contact emergency services, friends, or family members could save the life of a suicidal app user.

Although more than half of the apps (72.64%) were rated highly (between 4 and 5 stars), user reviews reveal significant issues. Therefore, quantitative ratings may not adequately reveal users’ experience and usability issues associated with mental health apps. An in-depth analysis of users’ qualitative reviews (comments), such as the one conducted in this article, is necessary to uncover app strengths and weaknesses, and the reasons users cease engaging with these applications. Therefore, analyzing user reviews is a suitable approach to understand app weaknesses and provide design recommendations.

Even though some mental health apps have high ratings, based on the users’ written comments, some users dislike these apps. For example, the Headspace app received 700 reviews and a rating 4.9, but users’ written comments include both positive and negative comments. Moreover, Howells et al.³¹ evaluated the effectiveness of the Headspace app and found that the app reduced depressive symptoms. However, it had a high attrition rate (41% of participants who used the app dropped out).

Finally, 67.9 percent of the mental health apps available in the app stores seem to be commercially motivated and heavily focused on app functionalities, with little or no attention to overall user experience and usability issues which significantly impact the effectiveness and user adoption.

We believe that developing a usable app will not be more expensive than developing an unusable app if developers have appropriate guidelines. Thus, we provide recommendations to guide developers and facilitate the development of apps that are usable and more effective without necessarily increasing the price.

Design recommendations

We analyzed user reviews of mental health apps to highlight their strengths and uncover weaknesses that may hinder their effectiveness and lead to high attrition rates from the user’s perspective. Based on our findings, in this section, we offer some recommendations for designing mental health applications to improve users’ adherence and engagement with apps:

Developers should employ the participatory design method during the design process and ensure that they involve users in the design process.³² Most issues and design requirements such as the need for landscape mode and offline capability can be suggested by users. They should also consider getting input from clinicians during the design and evaluation stage,³³ which will also increase the trust of the app.

Developers should conduct an extensive evaluation of the apps before it is made available to the public. Usability is an important factor that enhances users’ engagement and reduces attrition rates.

Developers should provide adaptive functionalities that allow users to adapt some app features such as the font size, font color, background, and layout to suit their preferences. This will enhance the overall usability and ensure a personalized experience for each user.

Developers should provide a variety of options, functionalities, and content that users can choose from such as a variety of meditations, activities, and challenge types to keep users engaged.

Developers should provide information and explanation of mental health disorders and the effectiveness of the methods used in the app to reduce their issues. Evidence of effectiveness, scientific backing, and expert endorsement may be necessary to increase app credibility and build user’s trust in the app.

Developers should increase app credibility and gain users’ trust by preserving users’ privacy, avoiding unnecessary ads, and providing accurate information and reliable results. Also, a lock feature to keep personal collected data secure should be provided. Personal information should not be accessed without permission and unnecessary information should not be requested.

Users should have some control over the app features and functionality. For example, users could have an option to create an account on the app without providing all of the sensitive personal information needed for detailed personalization. More information can be requested once users are familiar with the app and trust has been established. App designers can also design a limited app version that users can use without creating an account, perhaps with minimal functionalities, that does not require dedicated users’ information and allow the user to upgrade to a full version by creating an account and setting their profile when they feel the need. In general, apps should never access any user’s information that is not needed for the app such as contacts or access any information at all without explicit user’s permission.

Developers should provide customer service that responds to users and fixes frequent issues.

The app should provide a form of social support with anonymous communication. Also, access to a professional moderator and emergency support should be provided in case of depression or suicidal feelings.

The app should provide an option for users to back up, sync, and export their data themselves to avoid data loss. App developers can also back up user’s data for them and allow recovery in the event of data loss. However, designers should be careful not to violate the user’s privacy when backing up data. Therefore, to strike a balance, designers should provide an opportunity for users to consent to their data being backed up (opt-in) and disclose how all stored data are used and who has access to it.

An offline mode that allows users to use the mental health app in the emergency or panic situations should be available. Designers can catch and locally save user history and data during offline use and upload the data once the user goes online.

A quick and easy to use guide should be available. No matter how easy the app may seem, developers should never assume that everyone can use it without a guide. A user guide or “how to” should be provided in various forms to serve diverse audience needs—text, video, and picture illustration. Designers should also include responses to frequently asked questions as part of their app.

Designers should update the app regularly to fix any known issues, ensure compatibility with new technology, and enhance app features to promote user engagement. It is also in the interest of app designers to regularly analyze app reviews and make refinements and updates based on the comments.

Strengths and limitations

Analyzing publicly available user reviews of mental health apps on both Google Play and the App Store gives us access to rich and large evaluation data set from a diverse audience. This is the main strength of this type of method. From the large data set, we were able to determine which qualities most users liked and disliked in mental health apps. We also found this method of qualitatively analyzing user reviews to be very useful for obtaining insights into user experience and opinion, uncovering strengths and weaknesses of the apps.

Another strength of our study is that all the recommendations for mental health app design we derived from our findings can be generalized to any kind of mental health apps because we included different app categories such as meditation, tracking, social support, games, and CBT theory in our analysis. We also argue that some of the findings (such as the need for personalization, user control, usability issues) can also be used to inform other health app designs since they are generic.

However, there are some weaknesses regarding this method. For example, according to Nicholas et al.,²³ there might be important themes that have not been captured and researchers cannot know the preferences and needs of users with mental health disorders who have not used mental health apps. To reduce these weaknesses, we conducted a thematic analysis of a large number of users’ reviews. Moreover, our study focuses on users’ experience of mental health apps, so there is no need to know the needs and preferences for those who are not interested in using mental health apps. There is a possibility that people who write reviews on the Google Play or App Store are not representative of the wider population of app users. It is also possible that some people who do not have any mental health issues also used the app and provided evaluations. Nevertheless, the large number of user reviews in our method allowed us to capture more data and provided us with a more diverse opinion.

Another limitation of the work is that we do not know the percentages of users who stopped using the app because of app issues or other reasons. We also do not know how long the apps should be used, and we could not find any information regarding this in the literature. The apps provide no information on how long people with mental health issues are expected to use the app to notice some improvements. Also, most of the apps’ descriptions did not mention the expertise of the actual people behind the app. Therefore, we suggest that it is important to provide such information about app developers as part of the apps’ descriptions in both Google Play and the App Store. This is in line with the Persuasive System Design Model.³⁴

Future work

An interesting area for future work would be to investigate the relative and comparative effect of our identified app qualities on user engagement and determine why some qualities are more important for adherence than others. It is also interesting to examine the comparative effectiveness of various strategies used in apps (e.g. tracking. meditation, relaxation, feedback) with regard to promoting mental health.

Conclusion

Understanding users’ opinions are critical if we aim to design effective apps that will be adopted and used by the target audience. We conducted a qualitative study of 106 mental health apps from Apple’s App Store and Google Play and employed thematic analysis on 13,549 reviews to identify key themes. Specifically, we generated insights into the strengths and weaknesses of available mental health apps by uncovering the features/qualities of the apps that users liked, disliked, and suggested improvements. This review extends the literature by highlighting which qualities of mental health apps are especially important to users and should be considered when developing an app. In particular, our findings from this review reveal several aspects of app design that developers of mental health apps should consider when designing their app to improve user experience, usability, adherence, and hence overall effectiveness. First, developers should focus on the usability of the apps. Mental health application designers should conduct usability evaluations of both their initial and updated versions. Providing users with a variety of rich content, personalization functionalities, and allowing them to use the app without many restrictions are very important. Moreover, increasing the app credibility by offering some security features to protect user’s data, and providing regular updates and new features will increase users’ trust and decrease the high attrition rates currently experienced by mental health apps. Developers should also pay special attention to the quality of customer service and also provide an emergency contact. Finally, we offer some design recommendations for mental health apps that will improve usability, engage the user, and promote mental health based on our findings.

Supplemental Material

Appindex1 – Supplemental material for Insights from user reviews to improve mental health apps

Supplemental material, Appindex1 for Insights from user reviews to improve mental health apps by Felwah Alqahtani and Rita Orji in Health Informatics Journal

Footnotes

Acknowledgements

We thank Mohamed Salimian for his help in the primary analysis of the sample data and the reviewers for their insightful comments; this paper is much improved as a result.

Author contributions

The first author (F.A.) extracts available data (user reviews) and conducted the study. All authors contributed to preparing the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

There is no ethical approval for this research since the author used available data.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We thank the NSERC Discover Grant for funding Dr R.O.’s reaserch. We also gratefully acknowledge support from King Khalid University.

ORCID iD

Felwah Alqahtani

Supplemental material

Supplemental material for this article is available online.

References

Keyes

CLM

. Mental illness and/or mental health? Investigating axioms of the complete state model of health. J Consult Clin Psychol 2005; 73: 539–548.

Nakashima

Yamauchi

Kijima

, et al. Finding submodularity hidden in symmetric difference, 2017. https://arxiv.org/abs/1712.08721

Dirkzwager

AJE

Bramsen

van der Ploeg

. Social support, coping, life events, and posttraumatic stress symptoms among former peacekeepers: a prospective study. Person Individ Diff 2003; 34(8): 1545–1559.

Torous

Laura

. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry 2017; 74(5): 437–438.

Roepke

Jaffee

Riffle

, et al. Randomized controlled trial of superbetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J 2015; 4(3): 235–246.

Arean

Hallgren

Jordan

, et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res 2016; 18(12): e330.

Enock

Hofmann

McNally

RJ.

Attention bias modification training via smartphone to reduce social anxiety: a randomized, controlled multi-session experiment. Cogn Therap Res 2014; 38(2): 200–216.

Alqahtani

Orji

Usability issues in mental health applications. In: Proceedings of the UMAP’19 adjunct publication of the 27th conference on user modeling, adaptation and personalization, Larnaca, Cyprus, 9–12 June 2019, pp. 343–348. New York: ACM.

Coyle

Matthews

Design and evaluation guidelines for mental health technologies. Interact Comput 2010; 22(4): 243–252.

10.

Yamaguchi

Hashimoto

, et al. Is computerised CBT really helpful for adult depression?-A meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity. BMC Psychiatry 2013; 13: 113.

11.

Donker

Petrie

Proudfoot

, et al. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013; 15(11): e247.

12.

Demyttenaere

Bruffaerts

Posada-Villa

, et al. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA 2004; 291(21): 2581–2590.

13.

Bhugra Tasman

Pathare

Priebe

, et al. The WPA-lancet psychiatry commission on the future of psychiatry. Lancet Psychiat 2017; 4(10): 775–818.

14.

Faurholt-Jepsen

Frost

Ritz

, et al. Daily electronic self-monitoring in bipolar disorder using smartphones—the MONARCA I trial: a randomized, placebo-controlled, single-blind, parallel group trial. Psychol Med 2015; 45(13): 2691–2704.

15.

Knox

Lentini

Cummings

, et al. Game-based biofeedback for paediatric anxiety and depression. Ment Health Fam Med 2011; 8(3): 195–203.

16.

Torous

Nicholas

Larsen

, et al. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health 2018; 21(3): 116–119.

17.

Sarkar

Gourley

Lyles

, et al. Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 2016; 31(12): 1417–1426.

18.

Sunyaev

Dehling

Taylor

, et al. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc 2014; 22(e1): e28–e33.

19.

Nicholas

Boydell

Christensen

Beyond symptom monitoring: consumer needs for bipolar disorder self-management using smartphones. Eur Psychiatry 2017; 44: 210–216.

20.

Miner

Milstein

Schueller

, et al. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern Med 2016; 176(5): 619–625.

21.

Pagano

Maalej

. User feedback in the appstore: an empirical study. In: Proceedings of the 2013 21st IEEE international requirements engineering conference, Rio de Janeiro, 15–19 July 2013, pp. 125–134. New York: IEEE.

22.

Stawarz

Cox

Blandford

Don’t forget your pill! In: Proceedings of the 32nd annual ACM conference on Human factors in computing systems—CHI ’14, Toronto, ON, Canada, 26 April–1 May 2014, pp. 2269–2278. New York: IEEE.

23.

Nicholas

Fogarty

Boydell

, et al. The reviews are in: a qualitative content analysis of consumer perspectives on apps for bipolar disorder. J Med Internet Res 2017; 19(4): e105.

24.

Thach

. User’s perception on mental health applications. In: Proceedings of the 2018 5th NAFOSTED conference on information and computer science (NICS), Ho Chi Minh, Vietnam, 23–24 November 2018, pp. 47–52. New York: IEEE.

25.

Download app reviews from iTunes App Store amp; Google Play Heedzy. https://heedzy.com/ (accessed 27 February 2019).

26.

Stoyanov

Hides

Kavanagh

, et al. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015; 3(1): e27.

27.

Braun

Clarke

Using thematic analysis in psychology. Qualitative Research in Psychology 2007; 3(2): 77–101.

28.

Guest

MacQueen

Namey

. Introduction to applied thematic analysis. Applied Thematic Analysis 2012; 3: 20.

29.

de Alva Estrada Martinez

Wadley

Lederman

. It feels different from real life: users’ opinions of mobile applications for mental health. In: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, December 2015, pp. 598–602. ACM.

30.

Price

Sawyer

Harris

, et al. Usability evaluation of a mobile monitoring system to assess symptoms after a traumatic injury: a mixed-methods study. JMIR Ment Health 2016; 3(1): e3.

31.

Howells

Ivtzan

Eiroa-Orosa

FJ.

Putting the “app” in happiness: a randomised controlled trial of a smartphone-based mindfulness intervention to enhance wellbeing. J Happ Stud 2016; 17(1): 163–185.

32.

Fleming

Dixon

Frampton

, et al. A pragmatic randomized controlled trial of computerized CBT (SPARX) for symptoms of depression among adolescents excluded from mainstream education. Behav Cogn Psychother 2012; 40(5): 529–541.

33.

Dekker

Williams

AD.

The use of user-centered participatory design in serious games for anxiety and depression. Games Health J 2017; 6: 327–333.

34.

Harjumaa

Oinas-Kukkonen

Persuasive systems design: key issues, process model, and system features. Commun Assoc Inform Syst 2009; 241. DOI: 10.17705/1CAIS.02428.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB