Abstract
Methods: The Health App Review Tool has been designed to assess smart phone health apps according to their compatibility to individuals within the Alzheimer’s disease community. Specifically, app features and functions are characterized according to their appropriateness to the needs, abilities, and preferences of potential users. The Health App Review Tool is comprised of two components, the App and User Assessment; each component includes four complementary domains. Items in these domains can be compared between App and User assessments using a scoring key that will produce a match score. The score indicates the level of appropriateness in reference to the app’s ability to meet the user’s needs.
Introduction
Alzheimer’s disease (AD) continues to grow devastatingly pervasive among the older adult population, with a projected prevalence of 13.8 million in the United States by 2050. 1 Those living with AD typically present with a myriad of symptoms such as memory loss, functional decline, changes in behavior, and communication abilities. Assistance and support provided through informal caregivers (unpaid caregivers, often family members) is often required as AD progresses, and the symptoms associated with the disease grow more pronounced. Even with the support of an informal caregiver, the physical, emotional, and cognitive symptoms of AD can be challenging to manage. Day-to-day lifestyle variables can change or decline as AD progresses, eventually negating one’s ability to care for themselves. Basic activities of daily living, when altered or impaired can be very disruptive to the life of a caregiver and detrimental to the health of an individual with AD. For example, sleep disturbances have been reported as common and associated with distress in both the individual with AD and their caregiver. 2 The ancillary effects of AD, impacting not only the diagnosed individual but also negatively impacting health of the informal caregiver, 3 , 4 further highlights the need for effective interventions to support health and wellbeing in this population.
Smart phones are increasingly ubiquitous, even among older demographics. These readily available, off the shelf technologies may therefore serve as an optimal method of connecting AD caregivers to healthcare recourses, specifically through smart phone applications (apps). Many apps provide users with health management support through various functions that guide the user through lifestyle interventions or health maintenance practices. These might include reminders of when to eat or take medication, calendar reminders of upcoming appointments, memory aids to assist users in accurately following diet regimens, guided exercise routines, or sleep hygiene schedules. Apps that support sleep may monitor sleeps schedules, assist users in falling asleep using sleeps stories, or remind a user to take a sleep related medication. Any of these functions could be highly beneficial to address the afore mentioned sleep disturbances common in the AD population. However, each app—its functions, features, and usability—must be evaluated to assess whether or not it will be efficacious in addressing the specific needs of individual people with AD. Currently available app and software assessment tools5–9 lack attention to: (1) effectiveness of healthcare functions, (2) cognitively impaired users, and (3) caregiver users. The available assessments provide general feedback on apps. However, the rating or review of an application varies depending on the intended user. Therefore, we sought to design a tool to measure an app according to the needs, abilities, and preferences of an individual user.
Objective
The objective of this study was to develop the Health App Review Tool (HART) to (1) characterize apps, their features, functions, and contextual considerations, and (2) potential users from the AD population/AD caregiver population, and then (3) determine the level of match between apps and potential users based on these characterizations.
Methods
Development
The development of the HART was informed by existing mHealth assessment measures, literature on AD and AD caregiver population needs, clinical knowledge, and stakeholder data collected throughout focus groups. The HART was designed as a two-component assessment measuring (1) various usability and usefulness factors of apps, and (2) characteristics and ability considerations of AD/AD-caregiver users. Questions were designed to be answered by clinicians or healthcare providers who would be familiar with the general health factors, needs, and abilities of AD/AD-caregiver users. HART assessment items were designed to encompass the stakeholder perspectives, and were informed by existing mHealth and computer system measures.5–9
Preliminary focus groups were held to assess the stakeholder perspectives. Stakeholder groups of interest included both individuals with AD, their primary caregivers, and healthcare professionals. HART items were categorized according to themes extracted from both AD population and healthcare professional focus groups to include the following: usability, external use factors, specific needs, functions, usefulness, interaction and usage of app, adoption of app, external variables, and internal barriers (theme definitions and supporting data to be reported elsewhere) and additional items were added as needed to ensure that each theme was holistically addressed. Once all themes were sufficiently developed, the items for the app and user assessments were categorized into domains. The HART was then compared to pre-existing measures and literature. In sum, the HART was developed using a multi-step method that incorporated structuring and re-structuring HART content to maximize its ability to measure relevant variables, its user-centeredness, and usability.
Once a complete list of questions was developed for the HART measure, questions were organized into two components, the App Assessment and User Assessment. App assessment items cover variables such as the function of the app, evidence base of the app (if relevant), complexity of the app, cognitive burden of the app, and accessibility of the app. User assessment items cover topics such as the health and function needs of the intended user, their technology experience, technology support availability, education level, disease severity of the individual with AD, and physical accessibility needs. Rating scale categories between app items and user items were complementary. For example, items on the technological savvy of a client were rated on a 5 point scale ranging from low to high technological ability, while items on the complexity of the app were rated on a 5 point scale ranging from low to high complexity; these items could then be compared to determine if the technological savvy of the client was appropriately suited for the complexity of the app. Complementary App and User assessment domains are ultimately matched to across the following categories: (1) usefulness (will the app meet user specific needs), (2) complexity (considers app complexity and user preparedness to accommodate this complexity), (3) accessibility, and (4) external use factors. The final version of the HART included 106 items; the App assessment piece consisted of items 1–59, and the remaining 60–106 items formed the User assessment.
Scoring
The HART scoring system included a series of Likert scale questions spanning Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree; and dichotomous Yes–No response options. These scores were translated into 1–5 scale for Likert scores and either 0 or 4 for most dichotomous responses. Differences between comparable items on each assessment component can then be compared. For example: Auditory Ability Score of 4 − Auditory Accessibility Score of 1 = Difference Score 3. The difference between comparable items was taken to determine the level of “match.” This resulted in five categories, Perfect fit, Good fit, Fair fit, Poor fit, and Very Poor fit with a difference of zero, one, two, three, and four, respectively.
Some of the comparable items in the App and User assessments could be assessed through a simple subtraction equation to attain the difference between item scores while others required more complex calculations. For example, several items are rated such that a higher rating indicates that the app is more suited to accommodate those with poor technology abilities; however, the comparable user items are rated such that higher scores indicate higher technology capacity. In such cases, one of these scores would need to be reverse coded prior to calculating the difference. This specific example also poses an additional problem—greater technological ability should not lead to a “Poor” or “Very Poor” fit rating. An app’s accessible design may very well make it particularly suited for a user with greater needs, but it would not also make it poorly suited for a user with lesser needs. Instead, a more universal design approach was taken such that an app with high support in an area of great need yields a high score, while this same app would still yield at least a good score in the case of a user who has lower need, e.g. greater ability. To accomplish this, scoring for some items was both reversed and truncated so that they could not receive a score lower than three and the difference between the complementary item would then not exceed two (“Good” fit). In other words, the difference would never yield a “Poor” or “Very Poor” fit. This in essence created a neutral option.
All scoring calculations were designed as an Excel function so that a detailed scoring key could be developed and assessed. Calculations were then translated to SAS syntax to ensure that scoring could be completed in a reliable manner.
Discussion
The HART was conceptualized and developed as a technology-user matching tool, grounded in literature, clinical expertise, and stakeholder perspectives. As HART items were compiled, they were categorized according to relevant constructs and compared against the themes extracted from stakeholder focus groups in order to ensure that item development was both logical and holistic. Both individual HART items and collective constructs were repeatedly revised throughout the development process. The HART assesses apps specifically for their applicability to individual users, with the understanding that what may be useful and helpful for one individual may not be valuable to another.
The widespread social distancing measures implemented across the globe in response to COVID-19 have brought increased attention to the value of remotely delivered healthcare options. Health apps are a prime example of an intervention that can be implemented and utilized in the home context. The quickly expanding repository of more than 318,000 of health apps 10 has produced both high- and low-quality apps, not all of which are appropriate for use by AD and AD caregiver populations. Furthermore, the appropriateness of an app is an inconstant qualifier—meaning a specific app may be appropriate and beneficial for one user in the AD/AD caregiver community, while futile for another. The HART is designed to assess the appropriates of specific apps for specific users in the AD population. In doing this, the HART is intended to connect those with AD and their caregivers with health apps that will be both usable and useful to their individualized needs. Better ability to assess apps in a client-centered manner is anticipated to increase the use of health apps in the community dwelling AD population, and thereby, to improve health and quality of life for these individuals.
Limitations
The novel nature of the HART’s design can be viewed as a limitation in that it deviates from typical assessment measurement approaches. However, the increasing focus on individualized care propels us to develop better ways of assessing the match between individual needs and potential solutions. Therefore, we anticipate that the HART will not only be effective within the intended population but will serve as a model for future matching tools specific to other diagnostic populations.
Future directions
The HART was created to be both client centered and structurally sound. Future research should evaluate the HART through formal psychometric assessments. Specifically, confirmatory factor analysis should be used to assess the unidimensionality and factor loading of the HART. In addition, Rasch analysis should be used to assess the reliability, rating scale structure, and the fit of intended users to the HART.
Conclusion
This assessment has been systematically designed to assess factors relevant to the usability and usefulness of apps. This design improves upon existing mHealth and software measures by considering the technology variables in light of user variables rather than using a rigid scale of good-to-poor. The HART assesses apps specifically for their applicability to individual users, with the understanding that what may be useful and helpful for one individual may not be valuable to another.
Supplemental Material
sj-pdf-1-jrt-10.1177_2055668320938604 - Supplemental material for Health App Review Tool: Matching mobile apps to Alzheimer’s populations (HART Match)
Supplemental material, sj-pdf-1-jrt-10.1177_2055668320938604 for Health App Review Tool: Matching mobile apps to Alzheimer’s populations (HART Match) by Julie Faieta, Brittany N Hand, Mark Schmeler, James Onate and Carmen Digiovine in Journal of Rehabilitation and Assistive Technologies Engineering
Supplemental Material
sj-pdf-2-jrt-10.1177_2055668320938604 - Supplemental material for Health App Review Tool: Matching mobile apps to Alzheimer’s populations (HART Match)
Supplemental material, sj-pdf-2-jrt-10.1177_2055668320938604 for Health App Review Tool: Matching mobile apps to Alzheimer’s populations (HART Match) by Julie Faieta, Brittany N Hand, Mark Schmeler, James Onate and Carmen Digiovine in Journal of Rehabilitation and Assistive Technologies Engineering
Supplemental Material
sj-pdf-3-jrt-10.1177_2055668320938604 - Supplemental material for Health App Review Tool: Matching mobile apps to Alzheimer’s populations (HART Match)
Supplemental material, sj-pdf-3-jrt-10.1177_2055668320938604 for Health App Review Tool: Matching mobile apps to Alzheimer’s populations (HART Match) by Julie Faieta, Brittany N Hand, Mark Schmeler, James Onate and Carmen Digiovine in Journal of Rehabilitation and Assistive Technologies Engineering
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded through the Ohio State University Alumni Grants for Graduate Research and Scholarship and Rosita Schiller Award. This article content is revised from the first author’s dissertation which will eventually be available through Ohio Link.
Contributorship
JMF and CPD were primarily responsible for the development of the tool, and JMF and BH were primarily responsible for the scoring key described within this study. All authors contributed to this project in various phases. JMF drafted the manuscript which was then assessed and approved by all authors.
Acknowledgements
We would like to acknowledge Megan Ridder and Paige Goodrich for their valuable contribution to the HART project.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
