Abstract
Objective
Employing three cycles of Design Science Research (DSR) to develop a mobile app ‘ESSC (Excellent Self Supervised HIV Care)’ to improve self-management among people living with HIV (PLWH).
Methods
This study is based on the DSR framework comprising three iterative cycles. In the Relevance cycle, PLWH participated in a survey of mobile health (mHealth) experiences and needs. In the Rigor cycle, the information-motivation-behavioural skills (IMB) model was applied to foundations of the app, and HIV specialists verified the contents. Experts evaluated the heuristic system and the app quality with the Mobile Application Rating Scale (MARS). In the Design cycle, ESSC was built on the findings of the other two cycles, and end-users tested the usability using uMARS.
Results
The contents of the app were developed based on user requirements. The IMB model led ESSC to supplement motivational components for self-management, which built five functions: information contents; health life records including mental and sexual health; interactive counselling with healthcare providers; setting health goals after watching videos; and my page for self-reflection. To reduce social stigma and promote acceptance of the information-driven app, we created animated characters with neutral and bright features. The HIV specialists evaluated content validity as highly appropriate. The MARS score by the overall raters was between 3-acceptable and 4-good: functionality, 4.38; information, 4.12; aesthetics, 3.96; engagement, 3.37; and subjective quality, 3.25.
Conclusions
The DSR approach is effective for implementing usable and useful mHealth. The ESSC app would be feasible and contribute PLWH to retention in care.
Introduction
People living with HIV (PLWH) undergoing effective antiretroviral therapy (ART) have a zero chance of sexually transmitting the virus to others. 1 With global progress in HIV treatment, 90% of PLWH receiving treatment have achieved viral suppression by 2020. 2 The Joint United Nations Programme on HIV/AIDS targets to achieve ‘95-95-95’ by 2030 (95% of PLWHs know their HIV status, 95% of those who know their HIV status are receiving ART and 95% of people on ART have viral suppression) to end the AIDS epidemic. 3 In 2021, data from the Korean Disease Control and Prevention Agency showed that the rate of ART receiving and viral suppression was 95% and 96%, respectively, close to the set targets, with most PLWH in retention receiving care. 4 Retention in the care continuum is a lifelong goal 5 that requires practicing HIV self-management, such as ART adherence, regular clinic visits, monitoring laboratory parameters and managing HIV-related or non-HIV-related comorbidities. Although the scientific knowledge of ‘undetectable equals untransmittable (U = U)’ has contributed to changing perceptions of HIV as a chronic disease, HIV-related discrimination and exclusion remain strongly prevalent in healthcare settings. 6 People living with HIV have a higher prevalence of non-HIV-related comorbidities 7 and psychological problems 8 than the general population. Moreover, social determinants of health such as income, education and occupation impact poor retention in care. 9 As with other chronic diseases, multidimensional and diverse health resources are required; however, it is difficult for PLWH to access healthcare resources except in HIV-specific clinics. There are structural barriers within healthcare facilities where PLWH seek care as well as internal barriers, such as stigma and discrimination, which pose challenges to their attaining healthcare. 10
Globally, mobile health (mHealth) for PLWH has been continuously developed to increase engagement in healthcare for less time and at low costs. 11 This paper describes a project to develop the first mobile app that can be commercialized and used for PLWH in Korea. The app is designed to facilitate integrated HIV care by combining users’ needs, therapeutic goals and medical expertise. Design Science Research (DSR) is applied to address the complex behavioural elements of HIV self-management in mobile devices. Design science is driven by the desire to improve the user environment through information systems. In the field of information systems, DSR approaches implement not only the practical utility of users but also the credibility based on experience and expertise. Building on scientific knowledge and ensuring expertise contributes to the goal-oriented design of science research projects. 12 The results of applying DSR can obtain insight into why a system is needed in the users’ environment as well as a newly designed system. 13 The DSR framework was previously used for user-centred designs of mobile apps, such as HIV prevention for those at high risk among men who have sex with men (MSM), 14 reading healthcare providers’ (HCP) tuberculin skin test results, 15 and supporting patient drug adherence. 16 Design decisions such as system development, functional capabilities and information content are critical to achieving the goals of scientific research. 13 This study employs three cycles of DSR to develop a mobile app (‘Excellent Self Supervised HIV Care, ESSC’) for improving self-management among PLWH.
Methods
Overview
The ESSC app development was based on an iterative process for DSR of HIV self-management. This study utilized three cycles of DSR, namely, relevance, rigor and design.12,13 The cycles are based on three research domains: environment, knowledge base and DSR. 12 The first step was to understand the users’ needs and context. The relevance cycle of this study was derived from a requirement assessment survey. In the rigor cycle, the app components were organized based on scientific theory, with their validity being assessed by experts. 12 The design cycle involved building the artifacts including functions, contents and features on user requirements and expertise base. 15 To increase acceptance, we determined the aesthetic and design features considering the users’ characteristics. End-user usability of the preliminary version ensured user acceptability, including ease of use and understanding. The three cycles support the acceptance of information systems and improve high-quality designs. 12 Figure 1 presents an overview of the iterative process based on the DSR framework. The procedure for the development stages of the ESSC app is shown in Figure 2.
The Design Science Research framework applied to develop the ESSC app. Resource: Hevner AR. A three cycle view of design science research. Scand J Inf Syst 2007; 19: 87–92.
Procedure for the development stages of the ESSC app.
Relevance cycle
The relevance cycle commenced with identifying and representing the requirements, as well as defining the research output in the application context. 12 We conducted user needs assessments to identify the application requirements and potential opportunities to improve self-management. A cross-sectional survey was conducted between November 2021 and August 2022. Participants were eligible if they were 19 years of age or older and had been diagnosed with HIV for at least one month. In five hospitals, infectious disease physicians and HIV counselling nurses provided recruitment leaflets including a QR code to access the needs assessment survey. When PLWH visited the hospitals for regular check-ups, they could collect the pocket-sized leaflets. An online self-report survey was appropriate for autonomous participation and minimizing face-to-face interaction during the COVID-19 pandemic. A total of 243 PLWH participated, and all responses were used in the analysis (i.e., no missing data). The self-reported questionnaires consisted of five main sections: socio-demographics, personal HIV conditions, mHealth usage and requirements, healthcare service needs and self-efficacy for HIV management. The online questionnaire was administered after informed consent was obtained and anonymity was protected.
Socio-demographics
Data on the following socio-demographic variables were collected: age, gender, gender orientation, education level and economic status.
Personal HIV conditions
Personal HIV conditions were assessed based on the following: duration since diagnosis, ART adherence (which measures the number of days of medication missed in the last two weeks; self-reported adherence scores ranged from 0 to 100; 0 = not taking the prescribed medications, 100 = taking all the prescribed medications), CD4+ T cell count and viral load at a recent check-up.
Mobile health usage and app requirements
Three items were used to assess any health app user experience, which asked about device types; whether the respondents ever used the app; and if yes, which of the functions they used. App requirements consist of the elements of design, information, records, reminders and communication, comprising 15 items; participants self-reported their views on the need for an HIV health app and the related requirements using a 10-point Likert scale (from 1 = ‘not at all’ to 10 = ‘totally need’). The items were constructed by referring to previous studies on mHealth acceptance for PLWH,17,18 and Cronbach's alpha for this study was 0.832.
Healthcare service needs
Forty items regarding health service needs were identified through a literature review, researcher workshop and validity evaluation of HIV counselling nurses and PLWH. When the content validity of items was rated by four HIV counselling nurses and eight PLWH, the result of the average scale level (S-CVI) was 0.85, which was acceptable, and the item-level content validity (I-CVI) ranged from 0.38 to 1.0. Items with low validity were revised.19,20 The items comprised six domains: HIV management (seven items), ART care (six items), psychological healthcare (six items), sexual healthcare (four items), health promotion (nine items) and family and social life (eight items). Participants self-reported how much they needed the healthcare services using a 4-point Likert scale (from 1 = ‘not at all’ to 4 = ‘very much need’). Cronbach's alpha for the present study was 0.947, and the value for each domain was: HIV management, 0.787; ART care, 0.847; psychological healthcare, 0.848; sexual healthcare, 0.783; health promotion, 0.845; and family and social life, 0.815.
Self-efficacy for HIV management
The Korean version of the self-efficacy for HIV management scale 21 was used, which is a translation of Shively et al.'s 22 original questionnaire; its validity and reliability for PLWH in Korea have been tested and confirmed. It consists of six domains: managing depression/mood, managing medications, managing symptoms, managing fatigue, communicating with the HCP and getting support/help. Participants rated how confident they were in their ability to perform specific behaviours on a 10-point Likert scale (from 1 = ‘not sure at all’ to 10 = ‘totally sure’).
Rigor cycle
The rigor cycle provides the knowledge base to ensure that the design artifacts follow the grounding process and contribute to the research project. 12 We proceeded in four phases to build foundations and realize expertise in the product and system.
Phase 1: Scientific theories
The information-motivation-behavioural skills (IMB) 23 model states that behavioural-related information and motivation facilitate behavioural skills to affect behaviour adherence; this principle has been applied to ART adherence intervention for PLWH.23,24 The scientific theory is an appropriate foundation for the information-driven app of this study. Based on the IMB model, we matched the functionality of the app and discussed additional components that were not drawn from the findings based on user requirements.
Phase 2: Experience and expertise
The experiences and expertise of HIV specialists contributed to the evolution of the app to achieve treatment goals in the application domain of the research. Six HIV specialists evaluated the validity of the content of the ESSC. The HIV specialists, including two infectious disease physicians and four HIV counselling nurses with an average career duration of 13 years, were provided with ‘the content description book for the ESSC app’, which consisted of six domains and 47 health information items. They rated the I-CVI for whether the content was important and appropriate for delivery to PLWH. They not only rated the validity of the items but also revised the content to ensure clinical credibility.
Phase 3: Design systems
The design system was refined through heuristic evaluation. Five informatics experts assessed the heuristics for the prototype version. The heuristics evaluators were as follows: one professor with expertise in nursing informatics, one nursing professor with experience in mHealth development, two practitioners from digital healthcare companies and one expert with a major in mobile computing, who had an average career duration of 8.3 years in their respective fields. They were provided with a mobile phone with the prototype installed, prepared by the research team for seven days for individual use. They recorded the usability comments and severity score using Berinit's mobile heuristics tool. 25
Phase 4: Design products
The products were audited by experts for quality before PLWH use. Five informatics experts evaluated the multidimensional items of the Mobile App Rating Scale (MARS), 26 together with heuristics. After refining the products from informatics experts’ feedback, we asked five HIV specialists to rate the MARS. The HIV specialists had to self-register as the patients and use the prototype for seven days, and then provided their quality ratings and feedback to the research team.
Design cycle
The design cycle iterates between construction, evaluation and subsequent feedback for further development. 12 The requirements are driven from the relevance cycle, while the foundations and validations are drawn from the rigor cycle. To design highly usable artifacts, we incorporated findings from the other two cycles. In the design cycle, three sessions were implemented with an iterative process.
Design session 1: Functions of the app
The purpose of the ESSC app was to contribute to HIV self-management practices. From the needs assessment, we identified the requirement prioritization of PLWH for mHealth and healthcare services. This requirement should be connected with HIV management behaviour. We conducted research workshops to determine whether the IMB model was complete when the user's desired components were applied to the factors of IMB. Additionally, the domain-specific distribution of self-efficacy for HIV management allowed us to reflect on which aspect was lacking and how it could be improved. The content to be included in the app functions and represented to users was originally constructed in ‘the content description book for the ESSC app’.
Design session 2: Features of the app
The high acceptability by PLWH was as important as the functionality of the ESSC app. The majority of PLWH are sexual minorities (e.g., MSM), and customized content that reflects their experiences and affirms their identity is important to increase the usability and acceptability of the intervention. 27 Storytelling from a first-person perceptive makes it effective for users to understand and absorb learning. 28 We created characters representing PLWH and integrated scenarios to help them understand the importance of health behaviours and drive affirming action in the HIV care continuum. Informative animations that are inclusive of relatable characters in video format were used in mHealth for PLWH. 29 While adherence to HIV treatment is an important behaviour for PLWH, the risk of one's HIV status being disclosed to others must be addressed. 30 The app should achieve the highest levels of security and privacy, including preventing assumptions about HIV through images such as those of red ribbons. 31
Design session 3: User-evaluation
A preliminary version, which reflected the expert's rigor cycle, involved usability testing with five PLWH, who participated in the need assessments of the relevance cycle and expressed interest in the usability test. Five PLWH had never met the research team and were unaware of the ESSC app during the development procedure. To maintain the confidentiality of PLWH, they participated anonymously via non-face-to-face contact. Their participation was administered only after informed consent was obtained online. PWLH self-downloaded the ESSC app from Google Play or App Store and registered the app on their mobile. They used the app for seven days and then evaluated the uMARS 32 and provided feedback to the research team.
Statistical analysis
Descriptive statistics were used to investigate the distribution of participants’ characteristics, their mHealth app requirements and healthcare service needs. The MARS scores were reported as mean ± standard deviation (M ± SD). Cronbach's alpha was used to calculate the internal consistency of the app requirements and health service needs. SPSS Statistics Version 24.0 (IBM, Armonk, New York) was used for all data analyses.
Results
Requirement assessment analysis: Relevance cycle
All responses from the 243 PLWH were analyzed to investigate the distribution of requirements. Based on the results of app requirements, three prioritized app functions were identified: information, records/reminders and counselling with a HCP. The app should be installed and available on both Android and iOS mobile devices for full user coverage. Although most PLWH were part of retention in care and had good ART adherence, they reported a high need for information related to overall healthcare services. As a result of self-efficacy for HIV management, the mean score of domains such as getting support/help and managing depression/mood was low. Therefore, the ESSC app is aimed to improve self-efficacy for HIV management by facilitating the management of moods and encouraging getting support.
Socio-demographics
The mean age of the participants was 38.65 (±9.92) years. The sample comprised 92% male participants. Nearly two-thirds of PLWH had a college degree or higher (62.6%) education level, and more than half perceived themselves as having low-economic status (50.6%). A majority of PLWH were MSM (83.5%), including homosexual (67.9%) and bisexual (15.6%) persons.
Personal HIV conditions
There was an average of 8.58 (±7.10) years since their HIV diagnosis, and 83.1% of them had viral suppression (HIV viral load defined as <200 copies/mL). Most participants (71.2%) had 200 or more CD4+ T cell count, which is not classified as AIDS, and a quarter of participants (26.3%) did not remember their cell count. They self-reported a high ART adherence mean score of 96.72 (±6.42) on a 0–100 scale, as 7.8% of participants missed medication over two days in the past two weeks.
Mobile health usage and app requirements
The mobile devices included Android (58.8%) and iOS (37.5%), some used both (3.3%), and only one respondent did not know the type of device. Most PLWH (70.4%) had used mHealth with the following functions: seeking health information (45.7%), monitoring exercise behaviours (32.9%), reminders on clinic visits (26.3%), PLWH social networking (chatting, etc.) (18.1%), reminders on daily medications (16.5%), communicating with their HCP (14.8%) and recording laboratory results (9.9%). Mobile health app requirements on supporting the treatment and healthcare of PLWH are shown in Table 1. The desired components included evidence-based, trusted health information (9.61); security and privacy (9.50); easy-to-use content (8.97) and easy-to-understand content (8.89). In particular, participants expressed the need for counselling with medical staff (8.67) but reported a relatively low need to interact with other users (5.97).
Mobile health app requirements of PLWH (N = 243).
Abbreviations: M, Mean; SD, Standard deviation.
Healthcare service needs
The survey of healthcare service needs aimed to identify the information to be provided through the app. Overall, participants reported a high need regarding healthcare services with ‘very much need/need’ responses. The mean score of the 40 items was 3.33 (±0.41) (range 2.28–3.73) on a 4-point Likert scale. The high-scoring needs included ‘precautions for interaction between ART and other drugs’ (3.73) and ‘guidance on supporting medical expense’ (3.70). The results of all 40 items being ‘need or very much need’ led to our choosing all items as information content. It was also necessary to include all items because the information of the app needed to cover all subgroups of PLWH, such as those newly diagnosed people and with different gender orientations. In addition, Kim et al. developed a question prompt list (QPL) consisting of 12 items based on the information needs of PLWH in Korea. 33 After removing redundant items, we added seven of 12 items to make it easier for users to understand the information in the app environment. We constructed an information content list of 47 items (Table 2).
Item-level content validity (I-CVI) evaluation of information content by HIV specialists (N = 6).
Abbreviations: AIDS, acquired immunodeficiency syndrome; ART, antiretroviral therapy; CD4+ T, CD4 T lymphocyte; MSM, men who have sex with men; PEP, post-exposure prophylaxis; PLWH, people living with HIV; PrEP, pre-exposure prophylaxis; TB, tuberculosis.
Self-efficacy for HIV management
The mean score for self-efficacy for HIV disease management was 6.83 (±1.32) on a 10-point Likert scale; managing medications (8.79), communicating with the HCP (7.77), managing symptoms (7.44), managing fatigue (6.68), managing depression/mood (6.15) and getting support/help (4.31). Participants had low confidence in psychological and social-related HIV management behaviour, indicating a need for potential features of the app to empower mental health and reduce social stigma.
Early prototype organization: Rigor cycle
Scientific theories
Based on the IMB model, there are three fundamental determinants of health behaviour: information, motivation and behavioural skills. 23 The functions of information, record/reminder and counselling with HCP, which were drawn from the relevance cycle, were matched to the component of the IMB model. Comprehensive information including details of HIV and general health facilitates health behavioural skills, 24 which made up the 47 health information items. Motivation comprises personal and social motivation that are related to establishing a positive attitude towards HIV self-management based on individual belief and perceived social support. 23 While counselling with HCP includes the social motivation component,23,24 there was no function to encourage personal motivation. Individual awareness of meaningful change motivates engagement with the behaviour. Functions that enable motivational future orientation 24 and self-reflection 34 in-app devices enhance autonomous motivation. Therefore, we added the function of setting health goals, which orient an individual's future health and life and constructed self-reflection on ‘My page’. The component of behavioural skills consists of an individual's objective abilities that involve acquiring, accessible, self-cue and self-administering for HIV self-management. 23 As part of behavioural skills in this project, health life records/reminder was a practical function for managing HIV treatment and potential HIV-related health behaviour such as mental and sexual health. The components of the ESSC app applied to the IMB model are shown in Figure 3.
Components of the ESSC app based on the Information-Motivation-Behavioural skills model.
Experience and expertise
The six HIV specialists reviewed the content description book for the ESSC app, which was the deliverable of the scientific theory phase by the research team. The results of I-CVI evaluation for the importance of items and appropriateness of content are shown in Table 2. All scores were highly acceptable, indicating that the proportion of raters evaluated as quite/highly relevant for each item that was 0.80 or higher was acceptable. 35 One HIV specialist gave feedback regarding the difference between HIV and AIDS. Currently, almost PLWH are receiving ART and monitoring their CD4+ T cell count; therefore, including excessive details on HIV and AIDS cases depending on the CD4+ T cell count (i.e., HIV Stages 1, 2 and 3 (AIDS) defined as over 500, 200–499, and below 200 cells/mm3, respectively) can cause discrimination and stigmatization based on cell count. 36 Regarding precautions for emergency surgery, HIV specialists emphasized that HIV medical staff should not be the first point of contact. Owing to the fear of disclosure of HIV, PLWH want to receive a consultation with infectious physicians first. However, without wasting time, direct linkage to the hospital is more appropriate during emergencies. In psychological healthcare, HIV experts recommended practicing certain activities through app devices in addition to textual information. Reflecting on the feedback, we revised the content and constructed the weekly activity on perceiving individual mood and coping skills that were located in the function of health life records/reminders.
Prototype development: Design cycle
Functions of app
The five functions of the ESSC app were built while considering the other two cycles. The screenshots of the ESSC app functions are presented in Figure 4.
Screenshots of the ESSC app functions in the Design Science Research Cycle.
Information contents: A total of 47 components of health information in six domains were featured on the ‘Info’ screen. The integrated information was created to provide essential information about living a healthy life with HIV. For accuracy of healthcare information, the description contains verified literature such as HIV-related guidelines of the governments of South Korea, the US and European nations, and of the World Health Organization. Given that PLWH have a high need for trustworthy information, we attributed each screen of information to its source. Considerable information needs to be presented in a simple and accessible manner. To deliver intensive information, all content was designed to include brief descriptions and images, such as poster presentation templates. The title for each piece of information was framed as a question (i.e., would it be a problem if I do not notify others that I am infected?) referring to QPL. 33 Initially, when users logged in to the app, a pop-up screen showed four titles of the high-scoring information to increase user interest. Summary quizzes were included in each domain in the form of ‘O or X’ so that users could immediately check their answers.
Health life records/reminder: In the ‘life’ screen, the user records their health life details into four categories: today's medication, clinic visits, mental diary and sexual diary. In today's medication, they checked the boxes pertaining to daily ART medication adherence and the presence of uncomfortable symptoms after taking it. An alarm was set and a pop-up message was sent at the scheduled time, and if there was no record of medication, these were repeated in 5 min. Under clinic visits, the date of the clinic appointments and the results of CD4+ T cell count and viral load test were recorded. A week before the scheduled appointment, alarm and pop-up messages are sent to help the user prepare for the clinic visit. In the mental diary, four stages were constructed. First, they chose their strengths based on the previous week. The strengths section presented Peterson and Seligman's 24 strengths 37 with a description of the strengths. Second, the negative emotions that were experienced in the previous week were selected and they stated how they responded to those negative emotions. A message was provided to help users separate the negative emotion from the situation and avoid interpreting their situation negatively. Third, the positive emotions that were experienced in the previous week were recorded. This is a training to perceive positive emotions and experience them more often. Fourth, users recorded the total rate of their emotional score in the past week (−100 = as sad as I can imagine, 0 = not sad at all, 100 = as happy as I can imagine). Finally, upon completing the records in the mental diary, a pop-up message prompted users to use the counselling screen if the user wanted to talk about their emotion. Culturally, it was a challenge to document the sexual diary of PLWH in Korea. Nevertheless, it is a crucial aspect of the lives of PLWH. The user recorded the presence of sexual activity within the past week and checked the boxes for any difficulties they had in their sexual life (disclosing HIV to a partner, using condoms, fear of viral detection, believing that PLWH cannot have sex, etc.). Then, they determined safe sexual guidelines for PLWH.
Interactive counselling with HCP: The user could ask for various types of counselling, including medical information, personal concerns and app features. When a user left comments, the counsellor responded within 24 h. The counsellor was a nurse with more than 10 years of experience in research on PLWH. All functions of the ESSC app were anonymous and intended for personal use. There were no functions to communicate or share with other users. This reflected the low need score for interacting with others in the relevance cycle.
Setting health goals: The ESSC app focused on two goals across activities: adhering to lifelong HIV treatment and a mental health mindset. The user watched the ‘My Life's Healthcare Roadmap’ and ‘Resolutions for My Mental Health’ animations, which involve scenario-based storytelling. After watching, the user attempted to check the strategies listed in each section that enable HIV healthcare and mental health practice. Then, they recorded the individual healthcare roadmap for the next 10 years and the mindset for happiness.
My page for self-reflection: On ‘My page’, the user was able to identify a graphical and descriptive presentation of personal health records including medication adherence and mental and sexual health. The user's app engagement was shown as ‘fill pods with beans’. There were five pods labelled medication, mental health, sexual health, learning and my goals. Based on the activity of the app, related pod could be filled with beans. We used graphic data and animated characters, which could be presented visually and as personal summarized records.
Features of the app
ESSC stands for ‘Excellent Self-Supervised HIV Care’, and its phonetic meaning in Korean is ‘shrug’, a movement that raises the shoulders, a gesture used in Korea to signify confidence. This gesture is meant to encourage and empower PLWH who are practicing good self-management by daily ESSC access. Aesthetic images of ESSC were designed to be bright and cheerful based on the theme of helping PLWH self-management. The use of an animated character and careful descriptions make PLWH accessible and acceptable.
Animated character: Referring to the results of the requirements survey in the relevance cycle, the primary users of the ESSC app were considered as male, specifically MSM, in the age of 20–40 years old. They had low self-efficacy in mood management and getting support. We required a character that does not resemble a member of a specific population or group since PLWH should not be stigmatized. Animations were used as web-based interventions, since they ease comprehension, maintain attention and increase long-term memory. 38 A bright, gender-neutral image example was sent to the animation company, and after three revisions, the animated character was created. The character was the lead in the animation stories, delivering information as well as showing the integrated role model.
First-person narratives: Descriptions were sensitive to avoid stigmatizing terms and contained language that prioritized the person before the disease. 39 First-person narratives, in which the narrator expresses their view on topics or their experience, activate identification with the speaker and decrease resistance to cognitive acceptance.28,40 In addition, the use of ‘HIV’/‘AIDS’ terminology and phrasing such as ‘people living with HIV’ (i.e., from the third-person standpoint) was minimized in the app. The first-person narrative contributes to reducing the risk of HIV exposure to others by minimizing the use of ‘HIV’ throughout the app. The ESSC app also requires membership and administrator approval to use. Even after registering, users had to enter a password after logging off. Before logging in, the app appearance was designed to give no indication that it is an HIV management app.
Evaluation
Prototype evaluation: Rigor cycle
Design systems: The results of the heuristic evaluation of prototypes by five informatics experts are shown in Table 3. On a 5-point scale (score range: 0−4), a score of 3 or higher indicates a major usability problem, fixing which should be prioritized. 18 The experts provided five main comments that could be incorporated. First, the ‘home’ and ‘info’ screens were duplicated, to clarify the purpose of the app. The ESSC app is an information-driven app, aimed at providing verified health information to enable HIV self-management practice. Emphasizing information content in both ‘home’ and ‘info’ screens did not adhere to minimalist design. We revised the home screen to represent a brief message about the purpose of the app, instead of duplicated information icons. Second, one expert stated that the recent trend in inputting systems is to consider voice messages or picture uploads for ease of use. People living with HIV can be inconvenienced by unnecessarily including voice or pictures from HIV apps because they value the confidentiality of their profile on HIV-related sites. Third, users could not change the details from the initial medication settings. Daily medication should be set when members are registering; therefore, we added a button to change the settings. Fourth, the use of character images and the amount of content were inconsistent across screens. Information contents were denser and used fewer character images than on other screens. We partially attempted to modify this, but the inconsistency across screens remained. Finally, there was no direction in case of a program error. We corrected this using the counselling screen, which could communicate errors, individual questions, records and any feedback regarding the app. If any problem occurs, the 24/7 available phone number is presented on the screen.
Heuristic evaluation of prototype by informatics experts
Bertini's Mobile heuristics tool are assessed on a 5-point scale: 0 = no problem at all, 1 = Cosmetic problem only. Need not be fixed, 2 = Minor usability problem. Fixing be given low priority, 3 = Major usability problems. Fixing be given high priority, 4 = Usability catastrophes. Must be fixed.
Design products: The results of the MARS are presented in Table 4. The HIV specialists and informatics experts rated functionality and information as good: ‘The app is very concise and logically organized in terms of knowledge transfer’, ‘That is professional and contains information that patients usually wonder about, and it seems to be organized in functions that can actually help patients adhere in care’ and ‘The mind health part is interesting and impressive’. While engagement and subjective quality were rated as acceptable, there were concerns, such as ‘In terms of interest, I have some doubts about whether I will continue to use the app’ and ‘I am worried that they might be resistant to recording their sex life’. An HIV specialist gave the following suggestions for improvement: up-to-date information board including an injectable ART, human Monkeypox virus and so on and scale-up of the medication reminder settings that involved polymedication comorbidities with PLWH aging.
Results of the usability evaluation of the ESSC app on the MARS.
MARS categories are assessed on a 5-point scale: 1 = inadequate, 2 = poor, 3 = acceptable, 4 = good and 5 = excellent; b Two items of information were excluded from the ‘MARS-app user’, and only rated by HIV specialist and informatics experts.
User evaluation: Design cycle
User-usability: The results of uMARS are shown in Table 4. Five PLWH participated in the usability evaluation: men with a mean age of 36.2 years; three had used mHealth before and two had not. They had been diagnosed with HIV for an average of 9.8 years. Similar to the expert results, they gave good scores for functionality and information, but relatively low scores for engagement and subjective quality. As for the perceived impact, they rated it as good for knowledge, attitudes and behaviours towards self-management. They provided feedback for the seven days usability: ‘All content was good’, ‘reminder to record a mental diary at the end of the day’, ‘various medication alarm settings’, ‘more visual materials’ and ‘more information about symptoms and treatment of sexually transmitted diseases’. They provided feedback on the need for more fine reminder settings and more variety and content of information.
Discussion
In recent decades, mHealth has been spreading at a rapid pace due to its cost-effectiveness and ease of delivery. 11 However, researchers and HCPs agree that scientific validity is an important factor in the development process as well. 41 Three cycles of DRS enable suitable design science by emphasizing both pragmatic and scientific aspects. The relevance cycle underlines its pragmatic nature. Meanwhile, knowledge base, which includes scientific theories and the synergy between the rigor and relevance cycle, enables scientific design. 12 Based on the initial results of the relevance cycle, we set the IMB model as a theoretical framework for the design of ESSC app and performed cyclic and iterative development according to the components and process of three cycles of DRS.
Principal findings
The ESSC app for improving HIV self-management was developed for all PLWH. Specifically, the majority of users were in retention in the care phase. The Korean HIV epidemiology is similar to the demographic characteristics observed in the participants of this study, with a high proportion of PLWH being in their 20–40s and a majority of them being men at 93.6%. As of 2021, there are approximately 15,000 PLWH, and customized studies are poorly understood for subgroups of PLWH in Korea. 4 This study was the first development and evaluation study of an app available in Korean to support HIV self-management. This app targeted the complex behaviour of HIV self-management beyond viral suppression and required multi-components design for PLWH integrated care. Design Science Research, a problem-solving paradigm, could address the complex interactions among components. 13 Three cycles of DSR were applied to develop a mobile app that fulfils the goals of the project. Understand the PLWH environment in the relevance cycle and refine and complement it that contribute to HIV management through the knowledge-based rigor cycle. Expert validation was important in configuring content, systems and overall products. The process of verifying the groundings for the ESSC app was not only a task of the rigor cycle but also a meeting of the user's key requirements, as trusted content. The design cycle, which implements app construction by incorporating the other two cycles, was a core cycle in developing an acceptable app for the users. Based on user requirements and the IMB model approach, five functions were built: information contents, health life records/reminders, interactive counselling, setting health goals and my page for self-reflection. Animation and first-person narratives were specialized in-app features to increase the acceptability of health information. However, the prototype of the ESSC app was not well evaluated for its aesthetic and minimalist design in heuristic systems. In terms of product quality, experts were concerned about whether users could consistently engage in the app. Expert demonstrations provided practical improvements in the development process. Five end-users evaluated that the ESSC app had good functionality and information, but more fine medication reminder settings and more information contents were needed. Information content was not customized and not up-to-date, 11 which could reduce interest and decrease repeated use. In particular, there was no procedure of direct input from PLWH in designing the characters and features of the app. This seems to have led to lower scores in engagement and subjective quality of MARS. To improve engagement and increase acceptance, the usability data of end-users need to be scaled up and customized. We iteratively refined the app to meet users’ needs and to justify the purpose of the study. Three cycles, design scientific synthesis of knowledge base, and user requirements, provided an effective process for the ESSC app to develop into a potentially usable app for PLWH.
Mobile health solutions to improve HIV self-management
Mobile health is mostly used to search for healthcare information, and PLWH prefer trusted and easy-to-understand information. People living with HIV have poor health information resources, which are focused on hospital settings. 42 They avoid exposing their HIV status to others since they experience social stigma related to sexually transmitted infectious pathways, a high incidence rate of MSM, and a low incidence in Korea.6,43 Their reliable resource is limited to medical personnel, informing building an information-driven app first. The ESSC app comprises comprehensive 47 items that function as an easy-to-use source of information. We validated the app content on a knowledge basis through the review of national guidelines and verification by HIV specialists. As behaviour skills for HIV self-management, health life records/reminders were constructed as in-app activities. These were self-cues and potential activities for HIV self-management behaviour. The activities involved treatment adherence (medication adherence, retention in clinic visits) and mental and sexual health. In this study, they had lower mood management scores in self-efficacy for HIV management. People living with HIV have a higher incidence of mental health problems than the general population, leading to a barrier to HIV care continuum. 44 Resources for mental health screening and treatment should be easily accessible for PLWH. Although there were limitations on non-face-to-face contact in the ESSC app, mental health intervention was constructed as regular self-assessments, text messaging and counselling. Sexual health was a needed management component for PLWH and their partners 45 ; however, in Korea, sexual health had not yet become a common content for PLWH. The self-checklist for safe sexual activity was implemented through a sexual diary. These individual records were summarized on my page. My page and end of mental and sexual records were accessed with a link to the counselling screen. The ESSC app is designed to enable self-assessing HIV-induced health and gain access to ongoing the care continuum.
Limitations
The ESSC app focuses on autonomous requirements for community-dwelling PLWH to support with their HIV self-management and is not based on its use in specific hospital settings. This study had two dimensions in terms of limitations. The first is about this research involving a complicated and long step-by-step procedure. In the requirement survey, the research team constructed the questionnaire to determine the app components for PLWH and performed tests to verify the validity and reliability. However, stronger validity evidence gained through measurement of the tools is recommended for future studies. In addition, although the requirements were assessed by the quantitative survey, PLWH were not directly engaged in the process of designing characters and features of the app. The end-user evaluation of five PLWH over seven days was small a sample and short a duration for sufficient usability testing. Future studies that evaluate the effectiveness of health outcomes and understand user experiences are required to develop an enhanced version of the app. Second, the developed app function had a few limitations. The hospital records are not interfaced with the app. Although this avoids the risk of security breach of personal medical records, record operation by self-reporting is cumbersome owing to inputting clinic data and it could reduce the validity of records. Furthermore, most of the app functions are targeted at PLWH who are in retention in treatment. Falling-out-of-care people, such as those with no medical care appointments but with viral detection, were unable to provide accurate coverage for potential use. Mobile health for PLWH is not easily available in Korea, so the ESSC app prioritized usage by the PLWH population. Further studies are needed for content-customized apps that help manage HIV depending on the user's characteristics, such as age, treatment stage, gender and gender orientation.
Conclusions
The DSR approach is an effective framework for developing user-centred apps. In the relevance cycle, mHealth requirements of PLWH were identified as functions of information, treatment records and consultation with HCPs. People living with HIV are vulnerable to mental management and help-seeking towards HIV management. In the rigor cycle, we supplemented the functions of self-reflection and health goal setting with personal motivation factors appropriate to the behavioural science IMB model. Contents, systems and products of the prototype were verified by expertise to ensure grounding. In the design cycle, functions and features were implemented with animations and first-person narratives. We iteratively refined the app to be a usable and useful device of PLWH. The usability evaluation proved to be acceptable overall, especially good in terms of information and functionality. Three cycles, a synthesis of the knowledge base, and user needs provide an effective process for developing the ESSC app into an individual device that helps to improve HIV self-management.
Footnotes
Acknowledgements
The authors express our deep gratitude to the people living with HIV who participated in our study. We are grateful for the support of Animation Purple and Choazsoft Corporation in the process of developing the mobile app.
Contributorship
GSK, MSS and SAL researched literature and conceived the study. GSK, LK and SB were involved in protocol development, gaining ethical approval. JMK, JYY, JK, JYC and JPC were patient recruitment and audited protocol. LK wrote the first draft of the manuscript. GSK conducted a critical review and revision for important intellectual content. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Ethical approval
This study was approved by institutional reviews prior to the requirement survey [No. 4-2021-0910] and participation in development processes [No. 4-2022-1192].
Funding
The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by the National Research Foundation of Korea grant funded by the South Korean Government (Ministry of Science and ICT) (no. 2020R1A2C101081713) (principal investigator [PI]: Gwang Suk Kim) and the Faculty Research Grant from the Yonsei University College of Nursing (No. 6-2022-0034). The corresponding author received scholarships from the Brain Korea 21 FOUR Project funded by the National Research Foundation of Korea, Yonsei University College of Nursing.
Guarantor
GSK
