Sage Journals: Discover world-class research

Abstract

The development of eHealth applications for mental health interventions offers significant potential for enhancing accessibility and cost-effectiveness. This study investigates the usability and user-centered design of eHealth interventions, emphasizing the need to take neuropsychological considerations into account during the development process of an interface. We evaluated various eHealth applications using modified Nielsen’s Usability Heuristics. The findings demonstrate a lack of emphasis on the specific cognitive and perceptual needs of users with Alzheimer’s, schizophrenia, or major depressive disorder. Provided recommendations aim to make eHealth applications more user tailored. Our research highlights the need for a thoughtful approach to a design that ensures effective and accessible mental eHealth apps.

Keywords

mental health eHealth usability user-centered design heuristic assessment

Introduction

The field of eHealth applications offers promising avenues for mental health interventions, yet faces unique challenges in design and usability that impact their success and efficacy. eHealth can be defined as an electronic intervention with the intent to make healthcare low-cost and accessible (Deady et al., 2017). Usability and other user-centered design aspects are paramount in the development of any application. However, the stakes are higher when these applications are tailored for mental eHealth interventions, where utility and therapeutic effectiveness become key metrics of success. This research delves into the nuances of designing eHealth interventions for individuals with severe mental illnesses or neurodegenerative conditions, aiming to bridge the gap between general usability practices and the specific needs of this population.

Mental eHealth apps present a distinct set of design challenges, primarily due to the varied mental and perceptual limitations experienced by users with mental health conditions (Rotondi et al., 2017). Mental health apps allow individuals to track symptoms, access care, and train skills (Ondersma & Walters, 2020). Traditional application development does an adequate job of assessing the utility of eHealth interventions for general users. However, there exists a notable oversight in integrating neuropsychological considerations into the usability and accessibility of these applications. This oversight is critical since the cognitive and perceptual experiences of users, shaped by their mental health conditions, can drastically influence their interaction with the technology (Schouten et al., 2022). Clinicians and patients alike have found a need for eHealth apps that target specific mental health disorders and different applications of treatment types (Burger et al., 2020). It is in the best interest of mental health eHealth app providers to collaborate with patients to make the app as successful as possible for that targeted demographic (Leorin et al., 2019).

Alzheimer’s Disease

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder that negatively impacts memory, thinking, and basic day-to-day skills (National Institute on Aging, 2023; Pais et al., 2020). Regarding mental health eHealth, individuals with AD may interact with and perceive the app’s content in vastly different ways depending on their state. Another aspect of AD is vision and cognition, which is known to be severely impacted and will affect an individual’s ability to interact with the app (Rizzo et al., 2000).

Schizophrenia

Schizophrenia is characterized by delusions, hallucinations, disorganized or catatonic behavior, and other symptoms (American Psychiatric Association, 2022). Similar to those with AD, those with schizophrenia may interpret information within the app differently from those without these conditions. Users with mild cognitive impairment may require specialized app features to meet their unique needs which can also vary with time. For example, time perception is impacted for schizophrenia patients, which can negatively affect mental health eHealth interactions, especially if regular use is required for efficacy (Gómez et al., 2014).

Major Depressive Disorder

Major Depressive Disorder (MDD) is characterized by a change in function due to depressed mood and/or a loss of interest or pleasure in daily activities (American Psychiatric Association, 2022). Diagnosed individuals may experience a change in appetite or sleep patterns, suicidal ideation, and more. MDD patients experience cognitive dysfunction and have specific user needs for eHealth apps (Chakrabarty et al., 2016). Cognitive dysfunction includes impairment of attention, memory, processing speed, concentration, and learning (Zuckerman et al., 2018). Individuals with MDD seeking eHealth need an app developed with these symptoms in mind which adapts to any stage of their depression.

Current Study

The present research addressed these challenges by focusing on the usability and user-centered design of eHealth interventions for individuals with severe mental illnesses. It aimed to go beyond general usability principles, investigating how these applications can be designed to account for the perceptual, cognitive, and emotional experiences unique to this user base. The goal was to understand limitations in eHealth apps that could negatively affect individuals with mental health challenges and identify design strategies to accommodate their user experiences and needs.

By examining the intersection of mental health conditions and human-computer interaction (HCI), the research aim was to contribute to a more nuanced understanding of eHealth usability and provide evidence-based recommendations for developing eHealth interventions that are not only functional but also empathetic to the psychological and cognitive profiles of users with severe mental illnesses.

Method

To assess the usability and accessible design of neuropsychological eHealth interventions, heuristic analysis was utilized accounting for the unique requirements of neuropsychological populations. This approach involved an evaluation of three eHealth implementations, focusing on identifying and mitigating usability barriers that could impede user engagement. Nielsen’s 10 Usability Heuristics (Nielsen, 1994), adapted to incorporate accessibility considerations pertinent to neuropsychological conditions, was utilized for the study. This adaptation was informed by W3C Web Accessibility Initiative (WAI) and insights from neuropsychology, ensuring that the evaluation criteria were both comprehensive and relevant to our target user group.

A team of four evaluators assessed the three mental eHealth applications. The applications are available for both iPhone and Android smartphone applications. Two of the apps are online therapy treatments, while the third primarily provides pre-recorded therapy sessions. Evaluators have varied experience in human factors and clinical psychology. Evaluators independently evaluated the apps utilizing the modified usability heuristics. The evaluators identified usability and accessibility issues, subsequently presenting recommendations for user-centered design improvements, emphasizing changes that could significantly enhance user experience for individuals with neuropsychological challenges.

Results

The results of the heuristic analyses across all three applications are presented in Table 1 below. Interrater reliability for the assessments were fair agreement for app 1 (κ = .20) and slight agreement for app 2 (κ = .02), and app 3 (κ = .09). Independent of the agreement variability on heuristic assessment scores, evaluators identified some common themes when evaluating the three apps. These themes are presented in each heuristic section below.

Table 1.

Average Ratings Across All Evaluators for the eHealth Apps (1–3).

Heuristic	1	2	3
Visibility of system status	1.75	1.25	0.75
Match with real world	0.5	0	0
User control and freedom	2.75	1	1.5
Consistency and standards	0.5	0.5	1
Error prevention	2	0.75	2.25
Recognition vs. recall	2.25	1.75	1
Flexibility and efficiency	1.75	2	2.25
Esthetics and minimal	1.25	2	2.75
Recognition/recovery	0	1	0.75
Help and documentation	1.5	2.5	1.75

Visibility of System Status

Visibility of system status is the foundation of trust between the user and interface, thus, clear and effective feedback should be present at every interaction (Harley, 2018). However, this is especially important for people who have mental illnesses or neurodegenerative disorders since cognition and perception can be heavily impacted by these conditions (Web Accessibility Initiative, 2024). When cognition is affected, the way that information is stored, retrieved, and applied is impacted as well. Given that individuals experiencing mental illness are already managing their internal challenges, it is crucial to ensure the app’s user interface (UI) is straightforward and user-friendly to prevent overwhelming the patient (Good and Sam Sambhanthan, 2014).

Overall, the evaluated UIs did a good job at providing information, but there is room for improvement. In one application, there was a lack of real-time updates on therapist availability. Also, there was no clear communication on why an appointment could be canceled. Thus, confirmation messages, cancelation messages, and therapist availability updates require more accuracy and completeness. Also, the first interaction a user has with the UI is a demographics and mental health questionnaire, but does not provide a status bar to let the user know questionnaire progress. Improvements to these features would promote trust and reduce feelings of anxiety among patients whose cognition is affected because they will know exactly what to expect (Good and Sam Sambhanthan, 2014).

For conditions like AD, maintaining the user’s engagement while not adding to their cognitive load is imperative (Bogza et al., 2020). The content needs to be clear and detailed, using minimal layers of information and simple language. The design should be intuitive and user-friendly, with features such as pop-up windows and helpful tips, while avoiding drop-down menus and excessive scrolling.

Match Between System and Real World

Match between the system and the real world ensures that users can intuitively understand and interact with an interface by referencing familiar concepts and language (Kaley, 2018). This alignment is especially vital for individuals with mental illnesses or neurodegenerative disorders, whose perception and retrieval may be impaired (Web Accessibility Initiative, 2024). When retrieval and application of information is impaired, it is essential that interfaces present information in a straightforward and relatable manner.

The evaluated UIs performed very well in aligning with user expectations. However, at least one app requires improvements in making their information more accessible to people not familiar with psychological terms like “CBT.” Thus, a tool and glossary with brief descriptions of these technical terms would be beneficial.

Generally, across the UIs, the language used for navigation bars was user-friendly and included real world terminology. UIs suggested tasks in a recognizable way that reflected standard to-do lists. This design is ideal because individuals with cognitive disabilities find it challenging to understand and use inconsistent navigation mechanisms (Web Accessibility Initiative, 2024).

User Control and Freedom

The user control and freedom heuristic refers to allowing users to manage the system and adjust their interactions if they need to correct their actions (Rosala, 2020). For individuals with AD, MDD, and schizophrenia, interfaces should have consistent navigation, error recovery options like easy undo/redo, auto-save features, and clear exit functions to reduce confusion and frustration. These elements aid in preventing loss of progress and alleviate anxiety about making mistakes within patients.

The mental health intervention apps scored particularly low on this heuristic due to the intake form administered when first opening the app. This is one of the first interactions a user has with the interface, thus, it is imperative to make it accessible and user-friendly. The questionnaires did not allow for the users to go back once they had answered questions or exit until they got to the paywall. There was no way to go back to change an option after making the first selection. For example, when mistakenly selecting a suicide option, the user is directed to a page with crisis resources with no way to exit the page. This could exacerbate mood related symptoms associated with AD, MDD, or schizophrenia.

To improve the usability of the UI, there needs to be “Exit” or “Cancel” buttons to streamline the process of answering. Also, allowing for patients to rectify any mistakes that they may make is essential, thus, having a “Previous” option should be added to the interface. Lastly, patients should have the option to review all their answers and modify them, if needed, before submitting the survey that will determine every subsequent interaction with the application.

Consistency and Standards

The consistency and standards heuristic emphasizes the importance of uniformity in design, ensuring that similar elements and actions are consistent throughout the interface (Krause, 2021). Maintaining consistency in language, actions, and appearance will allow for a more positive and less confusing interaction with a system.

All the UI evaluated performed well on this heuristic. This aligns with the needs of individuals with cognitive disabilities, since inconsistent navigation mechanisms can negatively impact cognitive workload and HCI (Web Accessibility Initiative, 2024).

Error Prevention

Error prevention focuses on designing UI that help users avoid mistakes by providing clear and concise guidance before action is taken and offering easy ways to correct errors (Laubheimer, 2015). AD can make it difficult for individuals to identify and rectify mistakes (American Psychiatric Association, 2022). Therefore, explicit prompts and simplified interfaces are critical for a usable interface. Patients with MDD find it challenging to concentrate and stay motivated; thus, intuitive design, with easy-to-follow error prevention features and supportive guidance. Individuals with schizophrenia struggle with disorganized thoughts and distorted perceptions, leading to confusion or misunderstandings about feedback from systems. As a result, technology needs to provide more structured and repetitive instructions to help them avoid and correct errors effectively.

The individual needs of patients with an AD, MDD, or schizophrenia diagnosis need to be taken into account to create an interface that will cater to their tendencies and that way prevent errors from being made. This can be done by including mechanisms like confirmations before submitting actions and offering undo functions to minimize the impact of any errors. The evaluated interfaces performed moderately well, however, there is room for improvement. A button stating “Are you sure?” should be added for two-step confirmation while answering critical questions. While answering the demographic questions in one of the UIs, there should have been a review of all the answers given. It was also noted that all of the UIs had limited accessibility settings.

Recognition Rather than Recall

The recognition rather than recall heuristic focuses on making information and options easily visible and accessible to users, rather than depending on their ability to remember details (Budiu, 2014). Presenting clear cues and choices to AD, MDD, schizophrenia patients can immediately lighten their cognitive load and help avoid the strain of recalling information from memory. This principle is valuable for improving usability for people who face memory or cognitive challenges, as it streamlines interactions and reduces the need for users to actively remember details that pertain to the usability of the interface.

For AD, it is recommended that there be a setting added to the application which would not require the individual to insert their login information every time they use the interface. Individuals with a schizophrenia diagnosis can suffer from disorganized thinking and confusion, which requires all important information to be easily accessible and visible.

Changes can be made to the UIs to improve this area. Firstly, adding more reminders for the patients to indicate the progress they have made while using the intervention would improve HCI. Additionally, there should be a section where the patient or their caregivers can access past visits and notes that give them an overview of the topics covered during the last session. Also, one of the UI uses colors to organize and record the mood of the patient; however, this task would be difficult for a patient with cognitive disabilities (Web Accessibility Initiative, 2024). Thus, clear, non-ambiguous mood options should be available as well.

Flexibility and Efficiency of Use

The flexibility and efficiency of use heuristic can be defined as accessible shortcuts for experienced users, while allowing novice users to interact with the system without the shortcuts (Laubheimer, 2020). For individuals with mental health disorders starting to use eHealth interventions, the system should be straightforward and easy to use. Over time as they progress and continue using eHealth interventions, they can access shortcuts to streamline their experience. The interfaces evaluated did not allow for nearly any customizations and shortcuts to be made in the system. This would be an amazing tool for AD, MDD, and schizophrenia patients because the interface can be perfectly catered to their needs and differing cognitive abilities.

Furthermore, allowing for shortcuts and customizations in the system is a great addition for caregivers as well. Some examples could be a pin code access for caregivers to unlock certain features in the app that the patients would not be able to access otherwise, and writing specific reminder messages for their loved one such as indicating times they need to take their medications or when they need to do their hygiene routine and how to do it. Additionally, shortcuts can be made to minimize errors while using the interface since the decisions will be premade for the patients.

Aesthetic and Minimalist Design

The basis of the aesthetic and minimalist design heuristic is to refrain from excess units of information that interfere with the app’s interface (Fessenden, 2021). AD, schizophrenia, or MDD patients cannot have conflicting displays throughout the interface design. By prioritizing simplicity, adaptability, and supportive design elements, digital interfaces can significantly improve the interaction experience for users with these conditions.

The mental health apps that were evaluated performed well on this heuristic, meaning this coincides with the requirements necessary for individuals who are cognitively impaired. An overload of unimportant information can overwhelm them, leading to usability inefficacy of the system.

Error Recognition, Diagnosis, and Recovery

The error recognition, diagnosis, and recovery heuristic clarifies that error messages should be in clear language and easy for the user to find and interpret (Moran, 2019). Keeping the language of error codes in a manner that matches the real-world allows users of the system to easily understand the problem that caused the error. This makes it less daunting for individuals diagnosed with Alzheimer’s disease, schizophrenia, or major depressive disorder.

The evaluators rated this heuristic as above satisfactory for all three eHealth interventions assessed. This indicates that users with AD, schizophrenia, or MDD, should have little to no issues when there is an error while using the system.

Help and Documentation

The help and documentation heuristic refers to users having access to a help tool to navigate the app, that is easy to locate (Joyce, 2020). Specifically, users with AD, schizophrenia, or MDD should not have to expend mental energy locating help in order to use the system.

Across the systems that were evaluated, this heuristic scored adequate enough with minor recommendations for changes. This exhibits that requirements for users with cognitive impairments are mostly met. The main concerns were the lack of transparency about subscription necessary to access the system, as well as, no “Frequently Asked Questions (FAQ)” section. These resources allow users to make informed decisions regarding accessing the system. Easy access to help and navigation tools for users makes the system much more user friendly and less challenging.

Discussion

After completing the heuristic assessment for usability, accessibility, and needs of users with specific neuropsychological disorders, the following results were compiled. In general, the eHealth applications employed good usability and user-centered practices with varying efficacy. General usability problems existed across apps, with some apps exhibiting basic implementation of HCI practices with significant gaps in usability. Results were similar for accessibility.

In regards to the needs of the specific neuropsychological user populations, the apps often failed to consider the needs of users with mental illness. Specifically, the changes to perceptual, mental reasoning, reality interpretation, and cognition associated with severe mental health were not considered in the design of most evaluated eHealth applications. This leaves a considerable gap in the implementation of design practices which can impact a positive user experience for individuals with severe mental health disorders, limiting the efficacy of eHealth interventions.

The research underscores the importance of considering the perceptual, cognitive, and emotional experiences of users with mental health conditions throughout the design process. We propose that eHealth applications must be tailored to accommodate the diverse experiences and needs of this demographic, ensuring that the technology is not only accessible but also empathetic and supportive of their unique challenges. The study provides a path forward for the development of evidence-based recommendations for the development of eHealth interventions that are functional, engaging, and sensitive to the psychological profiles of users with severe mental illnesses.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Andrea S. Lefebre-Rivera

Fernando Montalvo

References

American Psychiatric Association. (2022). Diagnostic and statistical manual of mental disorders (5th ed., Text Revision). Pearson.

Bogza

Patry-Lebeau

Farmanova

Witteman

Elliott

Stolee

Hudon

Giguere

(2020). User-centered design and evaluation of a web-based decision aid for older adults living with mild cognitive impairment and their health care providers: Mixed methods study. Journal of Medical Internet Research, 22(8), e17406.

Budiu

(2014). Memory recognition and recall in user interfaces. Nielsen Norman Group.

Burger

Neerincx

M. A.

Brinkman

W. P.

(2020). Technological state of the art of electronic mental health interventions for major depressive disorder: Systematic literature review. Journal of Medical Internet Research, 22(1), e12599.

Chakrabarty

Hadjipavlou

Lam

R. W.

(2016). Cognitive dysfunction in major depressive disorder: Assessment, impact, and management. Focus: Journal of Life Long Learning in Psychiatry, 14(2), 194–206.

Deady

Choi

Calvo

R. A.

Glozier

Christensen

Harvey

S. B.

(2017). eHealth interventions for the prevention of depression and anxiety in the general population: A systematic review and meta-analysis. BMC Psychiatry, 17(1), e310.

Fessenden

(2021, January 24). Aesthetic and minimalist design (usability heuristic #8). Nielsen Norman Group. https://www.nngroup.com/articles/aesthetic-minimalist-design/

Gómez

Jesús Marín-Méndez

Molero

Atakan

Ortuño

(2014). Time perception networks and cognition in schizophrenia: A review and a proposal. Psychiatry Research, 220(3), 737–744.

Good

Sam Sambhanthan

(2014, June 22–27). Accessing web based health care and resources for mental health: Interface design considerations for people experiencing mental illness [Conference session]. Design, User Experience, and Usability. User Experience Design for Everyday Life Applications and Services: Third International Conference, DUXU 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, Proceedings, Part III (pp. 25–33). Springer International Publishing.

10.

Harley

(2018, June 3). Visibility of system status. Nielsen Norman Group. https://www.nngroup.com/articles/visibility-system-status/

11.

Joyce

(2020, December 13). Help and documentation: The 10th usability heuristic. Nielsen Norman Group. https://www.nngroup.com/articles/help-and-documentation/

12.

Kaley

(2018). Match between system and real world: 2nd usability heuristic explained. Nielsen Norman Group. https://www.nngroup.com/articles/match-system-real-world/

13.

Krause

(2021). Maintain consistency and adhere to standards (usability heuristic #4). Nielsen Norman Group.

14.

Laubheimer

(2015). Preventing user errors: Avoiding unconscious slips. Nielsen Norman Group.

15.

Laubheimer

(2020). Flexibility and efficiency of use: The 7th usability heuristic explained. Nielsen Norman Group.

16.

Leorin

Stella

Nugent

Cleland

Paggetti

(2019). The value of including people with dementia in the co-design of personalized eHealth technologies. Dementia and Geriatric Cognitive Disorders, 47(3), 164–175.

17.

Moran

(2019). Usability heuristic 9: Help users recognize, diagnose and recover from errors (video). Nielsen Norman Group.

18.

National Institute on Aging. (2023, April 5). Alzheimer’s disease fact sheet. https://www.nia.nih.gov/health/alzheimers-and-dementia/alzheimers-disease-fact-sheet#:~:text=Alzheimer

19.

Nielsen

(1994). 10 usability heuristics for user interface design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/

20.

Ondersma

S. J.

Walters

S. T.

(2020). Clinician’s guide to evaluating and developing eHealth interventions for mental health. Psychiatric Research and Clinical Practice, 2(1), 26–33.

21.

Pais

Martinez

Ribeiro

Loureiro

Fernandez

Valiengo

Canineu

Stella

Talib

Radanovic

Forlenza

O. V.

(2020). Early diagnosis and treatment of Alzheimer’s disease: new definitions and challenges. Brazilian Journal of Psychiatry, 42(4), 431–444.

22.

Rizzo

(2000). Vision and cognition in Alzheimer’s disease. Neuropsychologia, 38(8), 1157–1169.

23.

Rosala

(2020). User control and freedom. Nielsen Norman Group. https://www.nngroup.com/articles/user-control-and-freedom/

24.

Rotondi

A. J.

Spring

M. R.

Hanusa

B. H.

Eack

S. M.

Haas

G. L.

(2017). Designing ehealth applications to reduce cognitive effort for persons with severe mental illness: Page complexity, navigation simplicity, and comprehensibility. JMIR Human Factors, 4(1), e1.

25.

Schouten

S. E.

Kip

Dekkers

Deenik

Beerlage-de Jong

Ludden

G. D. S.

Kelders

S. M.

(2022). Best-practices for co-design processes involving people with severe mental illness for eMental health interventions: A qualitative multi-method approach. Design for Health, 6(3), 316–344.

26.

Web Accessibility Initiative. (2024). Cognitive and learning. https://www.w3.org/WAI/people-use-web/abilities-barriers/cognitive/#examples-of-barriers-for-people-with-cognitive-and-learning-disabilities

27.

Zuckerman

Pan

Park

Brietzke

Musial

Shariq

A. S.

Iacobucci

Yim