Digital health interventions for adolescents living with HIV in low- and middle-income countries: A narrative review and logic model

Abstract

The development of digital health interventions for adolescents living with HIV (ALHIV) in low- and middle-income countries (LMICs) remains poorly understood due to limited research. This narrative review applies a logic model to synthesize evidence on the development and adaptation of digital health interventions for ALHIV in LMICs. The review identifies studies involving interventions such as mobile apps or online support groups from databases including MEDLINE, PubMed, SCOPUS and gray literature. Grounded in the WHO framework for Youth-centered Digital Health Interventions, the model addresses health system challenges and links intervention components to process and health outcomes. Individual, family and environmental contextual factors are incorporated to enhance the model's relevance. This model serves as a guide for stakeholders, including designers, developers, implementers, researchers, and funders, in designing and evaluating digital interventions for ALHIV in resource-constrained settings, while highlighting key areas for future research and policy action.

Registration

N/A.

Tweetable abstract

A logic model for guiding the design of digital health interventions for adolescents living with HIV in low- & middle-income countries.

Keywords

adolescents living with HIV digital health logic model low- and middle-income countries technology

Background

Adolescence is the period in a person's life where dramatic change occurs within the individual (cognitively, emotionally, and physically) and how they interact with their social environment as they are finding their place in the world (Hashmi and Fayyaz, 2022). Apart from navigating the complexity of their developmental journey, adolescents also need to navigate the current time's volatility, uncertainty, complexity, and ambiguity (VUCA). Adolescents living with HIV (ALHIV) must manage living with a chronic illness that is highly stigmatized in addition to these challenges (Crowley and Rohwer, 2021; Enane et al., 2018; Meloni et al., 2020). This provides the impetus for focusing on adolescent health care that is empathetic to the needs of adolescents and considers their life context.

The fourth industrial revolution brought about rapid advancements in technology (Lee et al., 2018). Consequently, technology has become an inseparable part of the lives of adolescents. A review of health-related internet use amongst children and adolescents indicated that 82.8% in the age range of 11 to 18 years spend one to four hours per day online (Balogun et al., 2023; Lupton, 2021; Maitz et al., 2020; Park and Kwon, 2018). Adolescents over 15 years of age more frequently search the Internet for health information than those under 15 years; the Internet is also their preferred source of information (Lupton, 2021; Park and Kwon, 2018).

Digital health utilizes computer-based technologies (e.g., computers and software) and digital communication applications (e.g., social media and smartphone applications) (Food and Drug Administration [FDA], 2023); it includes m-health, which is the use of mobile wireless technologies for health (World Health Organization [WHO], 2020a). Digital platforms can be used to communicate health promotion or prevention information and reinforce disease self-management (Celik and Toruner, 2020). WHO (2018) categorizes digital health interventions according to how it is used in the healthcare system, for example, whether it is used by clients, healthcare providers, health system managers or data services. Client digital health interventions are further categorized into targeted client communication, untargeted client communication, client-to-client communication, personal health tracking, citizen-based reporting, on-demand information services and financial transactions (WHO, 2018). Each of the intervention categories is targeting a specific health system challenge. For example, for the challenge of poor retention in care, the intervention ‘targeted client communication’ could be used.

In the past 10 years, there has been a proliferation of digital health interventions for adolescents as indicated by the number of systematic reviews conducted to evaluate their effectiveness for health promotion, prevention, and management of mental health and chronic conditions, including HIV (Celik and Toruner, 2020; Crowley et al., 2023; Grist et al., 2019; He et al., 2021; Kouvari et al., 2022; Low and Manias, 2019; Navarra et al., 2017; Park et al., 2021). A systematic review by Celic and Toruner (Celik and Toruner, 2020) on the effect of technology-based programs on changing the health behaviors of adolescents found that 62.5% of adolescents were satisfied with the programs. Low and Manias (2019) found that adolescents and young adults with chronic diseases were receptive to receiving health information via a website or mobile app. However, most of the systematic reviews found that the evidence for digital health interventions to improve health outcomes is uncertain, as there is a lack of high-quality trials (Celik and Toruner, 2020; Crowley et al., 2023; Low and Manias, 2019). Further, most interventions were not theory-driven (Crowley et al., 2023). Although digital health intervention research is advancing globally, most studies have been conducted in high-income countries with a resultant paucity of evidence on the effectiveness of digital health interventions in low- and middle-income countries (LMICs) (Crowley et al., 2023; Lehtimaki et al., 2021).

Technological advances have brought about a plethora of technological designs to choose from, ranging from simple text messaging to smartphone applications, gamification, virtual reality and chatbots, but these remain generally unexplored in the context of HIV (Adams & Crowley, 2021; Hightow-Weidman et al., 2015). In 2020, WHO (2020a) released a guiding document on the planning, development and implementation of youth-centered digital health interventions. The guidelines set out principles of planning, for example, a needs analysis, determining whether digital health is the optimal intervention, and using a theory-driven approach. In the developmental phase, the content and delivery channel should be co-designed with adolescents and key stakeholders and a series of prototypes created. Implementation includes a phased launch, marketing and a monitoring and evaluation plan. However, the guiding document does not detail the various components of digital health interventions.

The COVID-19 pandemic has highlighted the need to increase access to health care for many marginalized population groups. Digital platforms provide an opportunity to address this and create opportunities to fully engage adolescents in care. Holly et al. (2021) argue that these platforms enhance adolescent well-being by increasing access to services, information, and opportunities for self-expression, education and communication (Holly et al., 2021). For ALHIV, these platforms offer benefits such as privacy, social networking, real-time monitoring and feedback (Adams & Crowley, 2021; Hightow-Weidman et al., 2015). Tarantino et al. (2022) reported in their study amongst young people living with HIV in Ghana, that young people use mobile phones for interaction with health care workers (e.g., appointments, lab results, clinic support groups, general discussions) and health tracking (e.g., adherence, physical activity, menstrual cycle). Most participants owned an Android smartphone and had daily access to the Internet. It is therefore clear that digital health interventions should be incorporated into adolescent health care. However, they are not a silver bullet and should be used to complement existing healthcare services rather than replace them (WHO, 2020).

Introduction to the study

Limited knowledge exists on the development, adoption, and adaptation of digital health interventions in low- and middle-income countries (LMICs), particularly for specific target groups. The rapid progression of artificial intelligence, exemplified by publicly accessible large language models (LLMs), underscores the importance of exploring their meaningful and secure integration into adolescent healthcare. This narrative review seeks to introduce a dynamic logic model for digital health interventions tailored to ALHIV in LMICs, intending to offer a foundational guide for forthcoming research endeavors. Logic models represent invaluable tools that enrich both research and program implementation by fostering consensus among stakeholders, enhancing planning and monitoring efforts, facilitating the replication of successful studies, and aiding in the comprehensive summary and synthesis of research findings (Smith et al., 2020).

Review questions

The following review questions were identified:

What specific health system and individual challenges faced by ALHIV can digital health interventions address?

Which types and components of digital health interventions are beneficial for ALHIV?

What outcomes can be used to monitor and evaluate the effectiveness of digital health interventions for ALHIV in LMICs?

How do individual, family and environmental contextual factors influence the development and implementation of digital health interventions for ALHIV in LMICs?

What is the role of stakeholders in the development and evaluation of digital health interventions for ALHIV?

Methods

Design

Narrative review methodology was suitable for this article as it is useful in exploring under-researched topics and presenting new ideas (Sukhera, 2022). Further, narrative reviews can be used to critically reflect on studies in the context of the wider body of literature to advance theoretical understanding (Greenhalgh et al., 2018). We followed the basic six steps of any review to maintain rigor, which are to formulate the research question and objectives, search for literature, screen articles for inclusion, assess the quality of the studies, extract and analyze the data (Paré & Kitsiou, 2017).

Search strategy

We searched databases such as MEDLINE, PubMed, SCOPUS, Science Direct, Sabinet and Google Scholar. Articles were also drawn from a previous systematic review of the authors. See Box 1 for the search strategy.

Box 1

search strategy

(“adolescent” OR “young people” OR “teen” OR “teenager”) AND

(“Information and Communications Technology” OR “ICT” OR “Technology” OR “Technology Enabled” OR “Technology based” OR “gaming” OR “social media” OR “ehealth” OR “mhealth” OR “whatsapp” OR “SMS” OR “mobile” OR “internet” OR “text message” OR “telemedicine”) AND

(“HIV” or “AIDS”) AND

(“Low-income countries” OR “Middle income countries”)

Source: (Crowley et al., 2023)

The database searches were updated to include articles up to April 2024, to ensure that all relevant studies were included.

Evidence selection

We included primary studies (n = 15) and reviews (n = 3) related to ALHIV and digital health; primary studies (n = 6) and reviews (n = 7) related to ALHIV; digital health intervention guidelines or recommendations from credible sources (n = 13); other studies related to digital health interventions (n = 8); recommendations made in other reviews on digital health interventions (n = 20 of which n = 13 is specific to adolescents); and gray literature or opinion papers (n = 2). Studies or documents that did not have applicability to LMICs and did not include adolescents as the primary population were excluded.

Data extraction

We extracted information on digital health interventions for ALHIV in LMICs that related to the purpose of the intervention, the description of the intervention, the outcomes assessed, and any feedback or recommendations made related to digital health interventions from other sources.

Data analysis

The extracted data was classified under conceptual categories including ALHIV/health system challenges, the components of digital health interventions, outcomes and contextual factors that influence the development and adaptation of digital health interventions (Aromataris et al., 2024). These conceptual categories were informed by the WHO Youth-centered digital health interventions framework (WHO, 2020a). Evidence was synthesized and contextualized into a logic model that can be applied to ALHIV in LMICs following the principles of advancing the field of logic models (Mills et al., 2019). Our logic model can be classified as a ‘type 4’ logic model (Mills et al., 2019) as it displays how the intervention interacts with the context to produce outcomes and can accommodate multiple intervention forms that are adaptable to the study context. Our interprofessional research team, composed mainly of young adults with expertise in nursing, public health, and psychology, validated the model. We chose not to employ consensus methods with external stakeholders to validate our logic model, as type 4 logic models are designed to be adaptable to different settings, which may hinder reaching agreements among stakeholders (Mills et al., 2019).

Results

Figure 1 represents the logic model of the components of digital health interventions for ALHIV and how they may impact outcomes. For this logic model, the focus is on how digital health interventions can be used by clients (adolescents).

Figure 1.

Logic model for digital health interventions for ALHIV.

Health system and individual challenges faced by adolescents living with HIV

The logic model starts with the problem or challenge that requires intervention or change. This involves identifying the challenge and determining if a digital health intervention offers a solution that could contribute to better outcomes. A landscape analysis or problem analysis should be conducted to identify which health needs the intervention should address. This is usually performed through a literature review as well as community consultations and formative research (WHO, 2020). The challenges of ALHIV include but are not limited to, poor mental health, isolation, risky behavior, non-adherence, low viral suppression rates and poor retention in care (Abubakar et al., 2016; Enane et al., 2018; Ritchwood et al., 2020; van Wyk et al., 2020). Health system challenges include a lack of adolescent-friendly services and health care workers not being trained in adolescent health care (Dahourou et al., 2017; van Wyk et al., 2020; Zanoni et al., 2019).

Digital health interventions

A digital health intervention should specify the purpose, underpinning theory, type of information, delivery platform and type of device. The Digital Implementation Investment Guide specify nine principles for digital development: (1) design with the user; (2) understand the existing ecosystem; (3) design for scale; (4) build for sustainability; (5) be data-driven; (6) use open standards, open data, open source and open innovation; (7) reuse and improve; (8) address privacy and security; and (9) be collaborative (WHO, 2020b).

There are various resources that intervention developers can use to engage with adolescents (Youth Cultural Development (YCD), 2023). The Youth-Centered Digital Toolkit recommends creating a safe space (genuine relationships, empowering youth), building personas (increasing your understanding of the needs of young people by creating scenarios that illustrate their needs), defining an opportunity (connecting the identified needs with the services and interventions), generate ideas (generate more than one idea and then prioritize), build prototypes (this can also be in the form of a flowchart), get real feedback (testing ideas with users) and do it again (repeat steps such as generating ideas to refine prototypes) (YCD, 2023).

Purpose

The purpose of the digital health intervention could be for information communication, to enhance self-management such as treatment-taking behaviors, or to provide peer support or healthcare worker support (Celik & Toruner, 2020; Crowley et al., 2023). It should be aligned with the health challenges identified. For example, if the challenge is social isolation, the purpose of the intervention may be to enhance peer support or to provide mental health support. The purpose of the intervention determines the other intervention components.

Underpinning theories

All digital health interventions must be theory-driven (WHO, 2020). The underpinning theory will guide the developers to choose the type of information, delivery platform, and technology design that will be aligned with the purpose of the intervention. There are various theories that researchers can choose from. Most digital health interventions are underpinned by social behavioral theories such as Cognitive Behavioral Theory, Social Learning Theory, Self-Efficacy Theory and the Theory of Planned Behavior (Cho et al., 2018). Researchers should map out the intervention's purpose and expected outcomes and then determine how the intervention activities, actions or behavior change methods will lead to the outcomes.

Type of information

The type of information delivered depends on the health needs to be addressed. The information needs should be evidence-based, tailored to the health needs, and address the specific concerns of ALHIV (the target group). The intervention designers need to decide if the information should be generally focused on health promotion in the area of health (e.g., sexual reproductive health, healthy lifestyles), school or life skills or if it should be disease-specific (e.g., adherence monitoring for ALHIV). Adolescents may prefer information that is not only focused on the disease (Allen et al., 2022) and may exhibit greater engagement with non-repetitive information that aligns with their specific interests, such as family planning, rather than content solely centered on treatment adherence (Shenderovich et al., 2021). To effectively promote self-management, information should encompass multifaceted aspects, encompassing role, emotional well-being, and medical considerations (Crowley and Rohwer, 2021). Tailoring information to developmental stages is imperative; for instance, younger adolescents may require distinct messaging compared to their older counterparts and youth (Bonnie et al., 2019).

As a first step, intervention developers need to identify if there is existing content that can be used or adapted or if new content should be developed. When using existing information, it should be adapted to be culturally and contextually appropriate. Young people or influencers can be used to advise on the content (WHO, 2020). Further, permission may be required to use the content. The type of information adolescents want may not be acceptable in a particular community or context. Another consideration is the language in which the content should be available. Information should be in a language that the ALHIV can understand and on acceptable reading and literacy levels. Translators must be from the local context to ensure that the translations are related to the spoken language and are not too ‘academic’. Adolescents may also prefer audio and/or audio-visual narratives of the information to written information (Demblon et al., 2023). Information or content should be engaging, attention-grabbing and culturally and contextually appropriate (Bhana et al., 2020). Advanced features could include personalized information with the use of artificial intelligence and machine learning or communication with trained chatbots (Nadarzynski et al., 2021).

Delivery platforms

Delivery platforms are central to digital health interventions. It involves how information or content will be delivered to the participants; the medium or channel that connects the user to the digital health intervention. The delivery platform is an important consideration when developing a digital health intervention, as it can affect user engagement, accessibility, and reach (Maddison et al., 2019). A well-designed delivery platform can increase the uptake and effectiveness of the digital health intervention, while a poorly chosen or poorly designed delivery platform can hinder adoption and impact (Klonoff, 2019).

These platforms need to address the health needs, be acceptable to adolescents and feasible to implement. A range of platforms have been used for ALHIV in LMICs, including application-based, web/computer-based, Short Message Service (SMS)/WhatsApp messaging, phone support/reminders and social media e.g., Facebook (Crowley et al., 2023; Dulli et al., 2020; Manby et al., 2022). Each platform has its strengths and limitations. The choice of delivery platform will depend on various factors, such as the purpose of the intervention, and the technical requirements.

When choosing a platform, considerations include the costs and requirements of the hosting platform, software licenses and the use of multiple languages. The platform needs to be suitable for the level of interactivity required by the intervention. The research team should develop a rough idea of what the user interface will look like and how the user should interact with it. User journeys can be developed to illustrate how a user will progress through the intervention. Further, personas of typical users can also be developed to illustrate how some users may benefit from the intervention. If the digital health intervention will require data, a needs analysis of internet connectivity and Wi-Fi access should be done. Mobile network operators could be engaged to learn how the intervention will work in the digital ecosystem. If buy-in from the local Departments/Ministries of Health is a future goal, early discussions with persons leading health informatics and health information systems are needed to discuss how the digital health intervention will feed data into the health information systems.

Each platform may have advantages over the other, for example, WhatsApp is a very accessible platform but may have issues related to anonymity. In Sub-Saharan Africa, HIV interventions have been primarily SMS-based (Casale et al., 2019; Manby et al., 2022). This may be due to a lack of access to smartphones and variable internet connectivity. However, basic SMS interventions may not be engaging enough for adolescents and lack advanced features (Tarantino et al., 2022). Smartphone applications may have better security and additional features such as gamification can be added but may be costly to develop and sustain; it also takes significant time and resources to develop an application. Meaningfully engaging adolescents when choosing a delivery platform is crucial as they are the end-users and have knowledge of the digital trends in communities. They should have decision-making power in the development process (WHO, 2020a). In addition, experts in digital technology can assist in choosing the correct platform for the intervention. There may be existing digital products that can be used, and one should avoid duplication of what is already available.

For interventions aimed at enhancing adherence, mobile applications employing gamification strategies prove effective, although factors like internet connectivity and access to compatible smartphones must be considered (Mulawa et al., 2023). The majority of youth express a preference for a blend of group support/chats and one-on-one assistance, with anonymity ranking among the key desired features (Tarantino et al., 2022). See Table 1 for a list of platforms, their purpose, main uses, advantages, disadvantages and prerequisites for effectiveness.

Table 1.

Aligning the delivery platform with the purpose of the intervention (author's own)

Delivery platform	Purpose	Main Uses	Advantages	Disadvantages	Prerequisites for Effectiveness
Simple SMS (Abiodun et al., 2021; Linnemayr et al., 2017; MacCarthy et al., 2020; Sánchez et al., 2021; Tarantino et al., 2022)	Improve access Promote health	Health education Reminders Self-care resources	Wide reach Cost-effective Timely No internet access required	Limited interactivity Limited functionality Technical challenges Cannot track health information Privacy/unintentional disclosure	Mobile phone access Literacy and language Cultural appropriateness Privacy and security
Advanced SMS (Hacking et al., 2019; Linnemayr et al., 2017; Stankievich et al., 2018)	Two-way communication Improve health outcomes Personalized support Promote healthy behaviors Increase engagement	Health education Consult with providers Self-care resources	Wide reach Personalized communication Real-time feedback Adherence to treatment Patient engagement No internet access required	Staffing costs Technical challenges Privacy	Trained staff Mobile phone access Literacy Cultural appropriateness Privacy and security
Telephone Calls or Telehealth (Hacking et al., 2019)	Provide remote care Personalized support Improve health outcomes Enhance access Increase engagement	Consult with providers Health education Medication management Mental health support Self-care resources	Wide reach Personalized communication Cost-effective Real-time feedback No internet access required	Staffing costs Technical challenges Language barriers	Trained staff Mobile phone access Language proficiency Cultural competency of staff Privacy and security
Messaging Apps (e.g., WhatsApp, Messenger, Mxit and Telegram) (Chory et al., 2022; Henwood et al., 2016; Stankievich et al., 2018)	Facilitate communication Improve health outcomes Personalized support Promote healthy behaviors Increase engagement Community development	Consult with providers Health education Medication management Mental health support Self-care resources Group support	Wide access Personalized communication Real-time feedback Convenient Cost-effective	Language barriers Technical challenges Requires smartphone and internet connection Potential privacy concerns – especially for groups e.g., being added to a group without knowledge	Access to smartphone Internet connectivity Trained staff Language proficiency Cultural competency of staff Privacy and security
Exclusively Web-Based Platforms (Ivanova et al., 2019)	Improve health outcomes Personalized support Promote healthy behaviors Increase engagement	Consult with providers Health education Medication management Mental health support Self-care resources	Accessible from any device Personalized Convenient for longer interactions Can provide more detailed information	Technical challenges Requires a device and internet connection Potential privacy concerns – especially for individuals who can only access through shared physical devices	Access to device Internet connectivity Privacy and security
Social Media (e.g., Facebook Groups) (Dulli et al., 2018; Dulli et al., 2020)	Facilitate communication Improve health outcomes Personalized support Promote healthy behaviors Increase engagement Community development	Consult with providers Health education Medication management Mental health support Self-care resources Group support	Large user base Multimedia capabilities Creating communities of support	Potential privacy concerns Technical challenges Limited control over content or information shared (should be monitored)	Trained Staff Access to the Internet and social media platforms Internet connectivity
Smartphone Apps (Mulawa et al., 2023)	Improve health outcomes Personalized support Promote healthy behaviors Increase engagement	Consult with providers Health education Medication management Mental health support Self-care resources	Personalized Interactive Multimedia capabilities	Costly to develop Requires smartphone and Internet connection Potential for app overload Digital/health literacy levels may affect effectiveness	Access to a smartphone that is compatible with the app Internet connection Need to consider health literacy levels
Gamification or game features/Game-based (Ahonkhai et al., 2021; Tarantino et al., 2022)	Improve health outcomes Personalized support Promote healthy behaviors Increase engagement Use of game design elements in non-game contexts – game stories, themes, characters, points and awards, leader boards, feedback	Health education Medication management Mental health support Self-care resources Symptom management Goal-setting Monitoring progress Medication adherence HIV education Social support and connectivity Social comparisons	Engaging, motivational Fun and play Use of social competition and promote healthy peer norms Providing feedback	Requires ongoing development and maintenance	Access to appropriate gaming platforms and content Finding a relatable theme/story and a point system that works Adaptable to diverse user preferences – optional participation in key features Offline access Use of code words for HIV-related terms

Technology design

The technology design refers to the overall architecture, functionality, and user interface of the digital health intervention. It encompasses elements such as user engagement strategies, interactivity, multimedia support, personalization, and data security (Stoumpos et al., 2023). The technology design is essentially the blueprint for the digital health intervention, outlining how it will work and what it will look like. When choosing a technology design for a digital health intervention, there are several factors to consider. Interactive designs that allow for user engagement and feedback tend to be preferred over non-interactive designs (Kunzler et al., 2019). Additionally, the use of human-like avatars and chatbots can increase engagement, while also providing 24/7 support. Developers should consider whether the intervention will be targeted towards individuals or groups, as this will determine the level of personalization needed. Gamification can be used to encourage healthy behaviors, while audiovisuals and videos can make complex health information more accessible (Mulawa et al., 2023). Offline access and the ability to work at one's own pace can increase accessibility. Customizable profiles and privacy settings can increase user comfort, while symptom and behavior tracking with feedback can help users stay on track. Finally, chat rooms and discussion boards can provide support and a sense of community. The timing, intensity, frequency, and duration of the intervention should also be tailored to the specific needs of the target population (Buckingham et al., 2019).

Type of device

The type of digital technology device will depend on the needs analysis and what types of devices adolescents use and have access to. This could include computers, tablets, or mobile phones/smartphones (Android or Apple). The type of device needs to be congruent with the delivery platform and technology design as the device's capabilities will influence the successful delivery of the intervention (Crowley et al., 2023). Adolescents tend to be heavy users of smartphones, making this an attractive option for digital health interventions. Smartphones provide the convenience of portability and accessibility, enabling users to access health information and support wherever they go. Additionally, smartphones are often seen as more private, which may increase the likelihood of engagement (Doyle et al., 2021). That said, adolescents in some low-resource settings might not have access to smartphones. Computer or laptop-based interventions may be more appropriate for interventions that require a larger screen, such as video-based education or interactive quizzes. These may also be accessible at schools. Tablets offer a balance between portability and screen size and may be useful in settings where smartphones are not available or appropriate. Ultimately, the choice of device will depend on the specific needs of the intervention and the target population. It is important to consider factors such as accessibility, privacy, and ease of use when choosing a device for a digital health intervention aimed at ALHIV. In addition, device ownership may place ALHIV at risk of crime and violence, depending on the setting (Nguyen et al., 2019).

Process outcomes

Process outcomes are on an individual and health system level. On an individual level, it can refer to acceptability, feasibility, perceived usefulness, preferences, functionality, stability, fidelity, and quality. On a health system/systems level, it refers to acceptability, feasibility, sustainability, and cost-effectiveness. It is important in the development stage of interventions that individual process outcomes such as acceptability and feasibility are iteratively assessed – e.g., first creating a prototype, obtaining preliminary feedback, revising, retesting, etc. (WHO, 2020a). A phased launch (launched first to a limited number of users) of the intervention will assist with obtaining feedback early and making necessary changes. With regards to feasibility, equity is an important aspect to consider, for example, if the intervention is m-health-based, access to mobile/smartphones and data availability should be considered. Sustainability means considering if the intervention will be able to stand on its own after the initial development and testing (McCool et al., 2020). Aspects to consider are therefore whether the intervention would be able to continue without any further funding and/or human resources or if continued funding would be required. Sustaining the intervention may require continuous editing, uploading of content, funding for trained coaches or moderators, etc. (Lokmic-Tomkins et al., 2022). Buy-in/ownership and funding from the government or other organizations might be needed to sustain the intervention or platform such as updating the content, maintaining, and managing the back end of the program or rolling it out on a larger scale (WHO, 2016). When developing the intervention, the potential for additions, upgrades or expansion should be kept in mind.

Qualitative and quantitative data can be collected to measure process outcomes. This can be in the form of descriptive surveys or interviews. If possible, data should be collected on the platform and included in the monitoring and evaluation plan. For example, the number of times a person accessed or used the platform, or the user queries or problems logged (Mulawa et al., 2023). This can be used to continuously evaluate the success of the intervention or improve its acceptability (WHO, 2020a).

Proximal and distal outcomes

Proximal outcomes

Proximal or short-term outcomes can be divided into those related to disease prevention and promotion and those related to disease self-management. Digital health interventions have shown promise in improving these outcomes and have been associated with various positive effects on individuals’ health behaviors and self-management. In terms of disease prevention and promotion, digital health interventions have been found to enhance knowledge and promote healthy behaviors such as physical activity and healthy eating (MacCarthy et al., 2020; Mulawa et al., 2023). For example, studies like Abiodun et al. (2021) demonstrated that digital health interventions can improve subjective measures of adherence among ALHIV. Dulli et al. (2018) found that participants in a Facebook support group intervention showed a significant increase in knowledge compared to the control group. Furthermore, digital health interventions can play a crucial role in reducing risky behaviors such as suicide, and drug/alcohol use (Champion et al., 2019). Targeted interventions, as illustrated by Chory et al. (2022), showcase the potential advantages of utilizing WhatsApp groups for enhancing treatment adherence, reducing stigma, and promoting mental and behavioral well-being among ALHIV. Regarding disease self-management, digital health interventions can improve perceived support, self-efficacy, self-management, adherence, and retention in care (Crowley et al., 2023). For instance, MacCarthy et al. (2020) showed that an SMS-based intervention improved adherence and overall quality of life among young people living with HIV. Dulli et al. also demonstrated that a structured support group intervention using a Facebook group led to increased access to support and information, decreased social isolation, and increased adherence to antiretroviral therapy.

Distal outcomes

Distal health or long-term outcomes are divided into individual health outcomes and systems-level outcomes. For the health system, digital health interventions have the potential to result in cost savings (Gentili et al., 2022; Gomes et al., 2022). Abiodun et al. (2021) showed promising health outcomes in Nigeria's STARTA Trial, using interactive text reminders to improve viral suppression among ALHIV, potentially reducing morbidity and health costs. In Chory et al. (2022), a study in Kenya using WhatsApp groups, there was a positive impact on mental health and adherence support for ALHIV. Moreover, interventions targeting ALHIV can significantly reduce viral load, lowering the risk of HIV transmission and other infections while improving immunity. In a study by Hacking et al. (2019) on the Virtual Mentors Program in South Africa, virtual mentorship effectively increased antiretroviral initiation and completion of viral load testing for ALHIV. These findings underscore the potential of virtual mentorship to enhance engagement in HIV care, leading to improved health outcomes.

Contextual factors influencing digital health interventions

Individual contextual factors include for example digital and reading literacy of the end-users. Generally, adolescents should have good digital literacy, but they may need specific instruction on the use of certain platforms (Meyers et al., 2013; Taba et al., 2022). Reading literacy is important if an adolescent is required to read health messages or participate in online activities. Information should be on a reading level e.g., Flesch-Kincaid Grade Level 7 or 8, that is easy to understand. The preferred language that they are comfortable in should also be considered. Some ALHIV may have developmental and cognitive delays due to the impact of HIV on their development (Dahmani et al., 2024; Sherr et al., 2018). This should be kept in mind when determining whether a digital health intervention can be meaningfully used. Individual aspects also include the types of devices adolescents are using in the context, whether they own/share such devices and the digital channels they are accessing. Providing adolescents with phones to deliver a digital health intervention is possible yet it reduces an intervention's cost-effectiveness and long-term sustainability (Tarantino et al., 2022). Adolescents of different ages and developmental stages may have different needs and preferences when it comes to digital health interventions. For example, younger adolescents may prefer more interactive and gamified digital health interventions, while older adolescents may prefer more mature and practical interventions (Hollis et al., 2017; WHO, 2020a).

Family-related factors include beliefs about technology and the socio-economic status of the family. Some families might not be supportive of adolescents owning or using phones, or the adolescent must share a phone with family members (Procentese et al., 2019). Some may not be able to afford the internet which may lead to poor connectivity issues. Other challenges include electricity outages leading to a failure to charge phone batteries (MacCarthy et al., 2020; Sánchez et al., 2021; WHO, 2020). The family dynamics could also impact an adolescent's ability to participate in a digital health intervention – for example, in some families, they are required to perform certain home tasks (Chory et al., 2022; Dulli et al., 2020; Hacking et al., 2019; WHO, 2020).

Environmental factors are geographical location, data or web access and cost, technology culture, legal age of consent, safety, security and privacy. An important environmental contextual factor is the need to promote safety, privacy, and ethical conduct. Users of digital health interventions should be guaranteed that their information will be protected, and they should have autonomy over how much information to share and to whom (WHO, 2020). The age of the adolescent may have legal implications in terms of obtaining informed consent as it relates to the protection of personal information legislation of each country. Parental consent is usually needed for adolescents under the age of 18 utilizing digital technologies that will collect personal information. In some cases, it is legally or ethically required to report certain disclosed information such as child abuse or risk of self-harm and in other cases it may not be able to guarantee complete confidentiality e.g., if social media platforms are used. Digital health interventions should have notifications to inform participants of these confidentiality limits but should have standard practices to ensure that users are informed about how their data will be collected, used and protected (Arora et al., 2014; McGraw and Mandl, 2021). Geographical location is another environmental factor that may determine, for example, the area's digital technology infrastructure and capabilities such as access to mobile networks or even the availability of electricity to charge phones. Although mobile networks are now more accessible and stable in LMIC, certain areas may still have variable connectivity (Vanden Abeele and Nguyen, 2022). Lastly, it is important to assess the general values and attitudes toward the use of digital technology by adolescents in a community and the social norms surrounding it. This also includes the safety of the environment e.g., the potential risk an adolescent may encounter carrying a mobile device in an unsafe community (WHO, 2020).

Discussion

There is a scarcity of research on digital health interventions for ALHIV and more research is needed to determine how the unique needs of this population can be met. We presented a comprehensive narrative review using a dynamic logical model for digital health interventions for ALHIV in LMICs that can inform intervention development, adoption or adaption. Intervention developers need to determine if and how digital interventions or technologies are appropriate to address the problem or challenge. They need to assess the available services for adolescents in their context and how a digital health intervention can complement existing health care or community services.

The development team should consider all the components of the digital health intervention as well as the contextual factors that can influence the implementation of the intervention. During the development process, they should create several prototypes and test them for functionality within the team. Heuristic application testing, also known as heuristic evaluation or heuristic usability evaluation, is a method used to assess the user interface and user experience design of a software application or website. It involves evaluating the system against a set of recognized usability principles or guidelines, known as heuristics (Gates, 2018; Senap & Ibrahim, 2019). When the team is confident that the digital health intervention is functional and looks polished, it can be ‘beta tested’ with ALHIV. Before testing, the team should ensure that all legal and ethical requirements are met, and potential risks mitigated. For example, compliance with laws for the protection of personal information and parental consent. A risk assessment should be done, and a mitigation plan created to address the risks. ALHIV should be encouraged to provide honest feedback. The feedback is then incorporated into the next version to ensure relevance, appeal, functionality, technical stability and usability.

Digital intervention development is a tedious process and should consider the evaluation of proximal and distal outcomes from the onset to test effectiveness and long-term sustainability.

Implications

The developed logic model can be used to guide the development, adaptation and testing of digital health interventions for ALHIV in LMIC, ensuring it is aligned with their health needs, considers contextual factors and includes standardized intervention components and outcome measures.

Limitations

The validity of the logic model must still be tested in further research. The review included a variety of evidence sources and used a flexible approach, which makes it less likely to be reproducible.

Significance

To the best of our knowledge, this is the first logic model that presents a synthesis of the evidence of the healthcare and ALHIV challenges, components of digital health interventions, outcomes and contextual influencing factors in LMICs.

Conclusion

In the era of the fourth industrial revolution and fast advances in technology such as artificial intelligence, digital health interventions should be considered as a key component of health service delivery. Therefore, healthcare workers, researchers and program implementers should know how to best use and integrate it into the healthcare system.

Footnotes

Acknowledgements

TC wrote the first draft manuscript and LW, CP and LT provided additional information and feedback. All the authors approved the final version of the manuscript.

Data availability

Not applicable.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

This was a literature review that is part of a larger study that was registered with the University of the Western Cape, South Africa (reference number: BM22/2/9). No ethical approval was required for this paper.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Talitha Crowley

About the authors

Talitha Crowley, PhD Nursing, is an Associate Professor at the University of the Western Cape. Her research areas are primary care, adolescent health, HIV, and self-management.

Leonie Weyers, BCur, RN, is a master's student at the University of the Western Cape and a Unit manager at Saartjie Baartman Centre, Cape Town. Her work focuses on managing a substance abuse unit.

Charné Petinger, MA Psychology, is a full-time PhD student in Public Health. Her areas of research are adolescent health, HIV, sexuality, and research psychology.

Lwandile Tokwe, BCur, MCur, RN, is a Senior Researcher at the University of Cape Town. His work focuses on HIV & co-morbidities in primary health care and qualitative research.

References

Abiodun

Ladi-Akinyemi

Olu-Abiodun

, et al. (2021) A single-blind, parallel design RCT to assess the effectiveness of SMS reminders in improving ART adherence among adolescents living with HIV (STARTA trial). Journal of Adolescent Health 68(4): 728–736.

Abubakar

Van de Vijver

FJR

Fischer

, et al. (2016) ‘Everyone has a secret they keep close to their hearts’: Challenges faced by adolescents living with HIV infection at the Kenyan coast. BMC Public Health 16(198): 197–197.

Adams

Crowley

(2021) Adolescent human immunodeficiency virus self-management: Needs of adolescents in the Eastern Cape. African Journal of Primary Health Care and Family Medicine 13(1): 1–9.

Ahonkhai

Pierce

Mbugua

, et al. (2021) PEERNaija: a gamified mHealth behavioral intervention to improve adherence to antiretroviral treatment among adolescents and young adults in Nigeria. Frontiers in Reproductive Health 3: 656507. DOI: https://doi.org/10.3389/frph.2021.656507.

Allen

Reda

Martin

, et al. (2022) The needs of adolescents and young adults with chronic illness: Results of a quality improvement survey. Children (Basel) 9(4): 500.

Aromataris

Lockwood

Porritt

et al, eds. (2024). JBI manual for evidence synthesis. JBI. [Online] Available at: https://synthesismanual.jbi.global. https://doi.org/10.46658/JBIMES-24-01

Arora

Yttri

Nilse

(2014) Privacy and security in Mobile health (mHealth) research. Alcohol Research 36(1): 143–151.

Balogun

October

Adebiyi

, et al. (2023) The influence of the 4th industrial revolution on family cohesion: A systematic review protocol. PLoS ONE 18(9): e0288954.

Bhana

Abas

Kelly

, et al. (2020) Mental health interventions for adolescents living with HIV or affected by HIV in low- and middle-income countries: Systematic review. BJPsych Open 6(5): e104–e104.

10.

Bonnie

Backes

Alegria

, et al. (2019) Fulfilling the promise of adolescence: Realizing opportunity for all youth. The Journal of Adolescent Health 65(4): 440–442.

11.

Buckingham

Williams

Morrissey

, et al. (2019) Mobile health interventions to promote physical activity and reduce sedentary behaviour in the workplace: A systematic review. Digital Health 5: 2055207619839883.

12.

Casale

Carlqvist

Cluver

(2019) Recent interventions to improve retention in HIV care and adherence to antiretroviral treatment among adolescents and youth: A systematic review. AIDS Patient Care and STDs 33(6): 237–252.

13.

Celik

Toruner

(2020) The effect of technology-based programmes on changing health behaviours of adolescents: Systematic review. Comprehensive Child and Adolescent Nursing 43(2): 92–110.

14.

Champion

Parmenter

McGowan

, et al. (2019) Effectiveness of school-based eHealth interventions to prevent multiple lifestyle risk behaviours among adolescents: A systematic review and meta-analysis. The Lancet. Digital Health 1(5): e206–e221.

15.

Cho

Y-M

Lee

Islam

SMS

, et al. (2018) Theories applied to m-health interventions for behavior change in low- and middle-income countries: A systematic review. Telemedicine Journal and e-Health 24(10): 727–741.

16.

Chory

Callen

Nyandiko

, et al. (2022) A pilot study of a Mobile intervention to support mental health and adherence among adolescents living with HIV in Western Kenya. AIDS and Behavior 26(1): 232–242.

17.

Crowley

Petinger

Nchendia

, et al. (2023) Effectiveness, acceptability and feasibility of technology-enabled health interventions for adolescents living with HIV in low- and middle-income countries: A systematic review. International Journal of Environmental Research and Public Health 20(3): 2464.

18.

Crowley

Rohwer

(2021) Self-management interventions for adolescents living with HIV: A systematic review. BMC Infectious Diseases 21(1): 431.

19.

Dahmani

Rakhmanina

Jianga

(2024). Cognitive impairment in children and adolescents living with perinatal HIV disease in the ART era: A meta-analysis. EClinicalMedicine, 72, 102602,

20.

Dahourou

Gautier-Lafaye

Teasdale

, et al. (2017) Transition from paediatric to adult care of adolescents living with HIV in sub-Saharan Africa: Challenges, youth-friendly models, and outcomes: Dahourou DL et al. Journal of the International AIDS Society 20: 21528.

21.

Demblon

M-C

Bicknell

Aufegger

(2023) Systematic review of the development and effectiveness of digital health information interventions, compared with usual care, in supporting patient preparation for paediatric hospital care, and the impact on their health outcomes. Frontiers in Health Services 3: 1103624.

22.

Doyle

Bandason

Dauya

, et al. (2021) Mobile phone access and implications for digital health interventions among adolescents and young adults in Zimbabwe: Cross-sectional survey. JMIR mHealth and UHealth 9(1): e21244–e21244.

23.

Dulli

Ridgeway

Packer

, et al. (2018) An online support group intervention for adolescents living with HIV in Nigeria: A Pre-post test study. JMIR Public Health and Surveillance 4(4): e12397–e12397.

24.

Dulli

Ridgeway

Packer

, et al. (2020) A social Media-based support group for youth living with HIV in Nigeria (SMART connections): Randomized controlled trial. Journal of Medical Internet Research 22(6): e18343.

25.

Enane

Vreeman

Foster

(2018) Retention and adherence: Global challenges for the long-term care of adolescents and young adults living with HIV. Current Opinion in Hiv and Aids 13(3): 212–219.

26.

FDA (2023) What is Digital Health? Available at: https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health (accessed 9 May).

27.

Gates

(2018) Toward valuing with critical systems heuristics. The American Journal of Evaluation 39(2): 201–220.

28.

Gentili

Failla

Melnyk

, et al. (2022) The cost-effectiveness of digital health interventions: A systematic review of the literature. Frontiers in Public Health 10: 787135–787135.

29.

Gomes

Murray

Raftery

(2022) Economic evaluation of digital health interventions: Methodological issues and recommendations for practice. PharmacoEconomics 40(4): 367–378.

30.

Greenhalgh

Thorne

Malterud

(2018) Time to challenge the spurious hierarchy of systematic over narrative reviews? European Journal of Clinical Investigation 48(6): e12931.

31.

Grist

Croker

Denne

, et al. (2019) Technology delivered interventions for depression and anxiety in children and adolescents: A systematic review and meta-analysis. Clinical Child and Family Psychology Review 22(2): 147–171.

32.

Hacking

Mgengwana-Mbakaza

Cassidy

, et al. (2019) Peer mentorship via Mobile phones for newly diagnosed HIV-positive youths in clinic care in Khayelitsha, South Africa: Mixed methods study. Journal of Medical Internet Research 21(12): e14012–e14012.

33.

Hashmi

Fayyaz

(2022) Adolescence and academic well-being: Parents, teachers, and Students’ perceptions. Journal of Education and Educational Development. 9(1): 27–47.

34.

, et al. (2021) Effects of smartphone-based interventions on physical activity in children and adolescents: Systematic review and meta-analysis. JMIR mHealth and UHealth 9(2): e22601–e22601.

35.

Henwood

Patten

Barnett

, et al. (2016) Acceptability and use of a virtual support group for HIV-positive youth in Khayelitsha, Cape Town using the MXit social networking platform. AIDS care 28(7): 898–903.

36.

Hightow-Weidman

Muessig

Bauermeister

, et al. (2015) Youth, technology, and HIV: Recent advances and future directions. Current HIV/AIDS Reports 12(4): 500–515.

37.

Hollis

Falconer

Martin

, et al. (2017) Annual research review: Digital health interventions for children and young people with mental health problems – a systematic and meta-review. Journal of Child Psychology and Psychiatry 58(4): 474–503.

38.

Holly

Brian Li

Agrawal

, et al. (2021) Opportunities and threats for adolescent wellbeing provided by digital transformations. Adolescent Well-Being: Background Papers for Multi-Stakeholder Consultations: Partnership for Maternal Newborn and Child Health: 1–18.

39.

Ivanova

Wambua

Mwaisaka

, et al. (2019) Evaluation of the ELIMIKA pilot project: Improving ART adherence among HIV positive youth using an eHealth intervention in Mombasa, Kenya. African Journal of Reproductive Health 23(1): 100–110.

40.

Klonoff

(2019) Behavioral theory: The missing ingredient for digital health tools to change behavior and increase adherence. Journal of Diabetes Science and Technology 13(2): 276–281.

41.

Kouvari

Karipidou

Tsiampalis

, et al. (2022) Digital health interventions for weight management in children and adolescents: Systematic review and meta-analysis. Journal of Medical Internet Research 24(2): e30675–e30675.

42.

Kunzler

Mishra

Kramer

, et al. (2019) Exploring the State-of-Receptivity for mHealth Interventions. Proc ACM Interact Mob Wearable Ubiquitous Technol 3(4): 140.

43.

Lee

Yun

J. J.

Pyka

, ,& et al. (2018). How to respond to the fourth industrial revolution, or the second information technology revolution? Dynamic new combinations between technology, market, and society through open innovation. Journal of Open Innovation: Technology, Market, and Complexity, 4(3), 21,

44.

Lehtimaki

Martic

Wahl

, et al. (2021) Evidence on digital mental health interventions for adolescents and young people: Systematic overview. JMIR Mental Health 8(4): e25847–e25847.

45.

Linnemayr

Huang

Luoto

, et al. (2017) Text messaging for improving antiretroviral therapy adherence: No effects after 1 year in a randomized controlled trial among adolescents and young adults. American Journal of Public Health (1971) 107(12): 1944–1950.

46.

Lokmic-Tomkins

Davies

Block

, et al. (2022) Assessing the carbon footprint of digital health interventions: A scoping review. Journal of the American Medical Informatics Association 29(12): 2128–2139.

47.

Low

Manias

(2019) Use of technology-based tools to support adolescents and young adults with chronic disease: Systematic review and meta-analysis. JMIR mHealth and UHealth 7(7): e12042–e12042.

48.

Lupton

(2021) Young people’s use of digital health technologies in the global north: Narrative review. Journal of Medical Internet Research 23(1): e18286.

49.

MacCarthy

Wagner

Mendoza-Graf

, et al. (2020) A randomized controlled trial study of the acceptability, feasibility, and preliminary impact of SITA (SMS as an incentive to adhere): A mobile technology-based intervention informed by behavioral economics to improve ART adherence among youth in Uganda. BMC Infectious Diseases 20(1): 173–173.

50.

Maddison

Rawstorn

Shariful Islam

, et al. (2019) Mhealth interventions for exercise and risk factor modification in cardiovascular disease. Exercise and Sport Sciences Reviews 47(2): 86–90.

51.

Maitz

Sendlhofer

, et al. (2020) Internet-based health information-seeking behavior of students aged 12 to 14 years: Mixed methods study. Journal of Medical Internet Research 22(5): e16281.

52.

Manby

Aicken

Delgrange

, et al. (2022) Effectiveness of eHealth interventions for HIV prevention and management in Sub-Saharan Africa: Systematic review and meta-analyses. AIDS and Behavior 26(2): 457–469.

53.

McCool

Dobson

Muinga

, et al. (2020) Factors influencing the sustainability of digital health interventions in low-resource settings: Lessons from five countries. Journal of Global Health 10(2): 020396–020396.

54.

McGraw

Mandl

(2021) Privacy protections to encourage use of health-relevant digital data in a learning health system. NPJ digital Medicine 4(1): 2.

55.

Meloni

Agaba

Chang

, et al. (2020) Longitudinal evaluation of adherence, retention, and transition patterns of adolescents living with HIV in Nigeria. PLOS ONE 15(7): e0236801.

56.

Meyers

Erickson

Small

(2013) Digital literacy and informal learning environments: An introduction. Learning, Media and Technology 38(4): 355–367.

57.

Mills

Lawton

Sheard

(2019) Advancing complexity science in healthcare research: The logic of logic models. BMC Medical Research Methodology 19(1): 55–55.

58.

Mulawa

Mtukushe

Knippler

, et al. (2023) Supporting adolescents with HIV in South Africa through an adherence-supporting app: Mixed methods Beta-testing study. JMIR Formative Research 7: e47575.

59.

Nadarzynski

Puentes

Pawlak

, et al. (2021) Barriers and facilitators to engagement with artificial intelligence (AI)-based chatbots for sexual and reproductive health advice: A qualitative analysis. Sexual Health 18: 385–393.

60.

Navarra

A-MD

Gwadz

Whittemore

, et al. (2017) Health technology-enabled interventions for adherence support and retention in care among US HIV-infected adolescents and young adults: An integrative review. AIDS and Behavior 21(11): 3154.

61.

Nguyen

D-N

Kumar

, et al. (2019) Cyber security in parallel and distributed computing: concepts, techniques, applications and case studies. Hoboken, NJ: Scrivener Publishing/Wiley.

62.

Paré

Kitsiou

(2017) Chapter 9 Methods for Literature Reviews. In: Lau

Kuziemsky

(eds) Handbook of eHealth Evaluation: An Evidence-based Approach. Victoria: University of Victoria.BC: University of Victoria. Available at: https://www.ncbi.nlm.nih.gov/books/NBK481583/

63.

Park Kwon (2018) Health-related internet use by children and adolescents: Systematic review. Journal of Medical Internet Research 20(4): e120–e120.

64.

Park

M-J

Seo

Y-G

(2021) Effectiveness of information and communication technology on obesity in childhood and adolescence: Systematic review and meta-analysis. Journal of Medical Internet Research 23(11): e29003–e29003.

65.

Procentese

Gatti

Di Napoli

(2019) Families and social media use: The role of parents’ perceptions about social media impact on family systems in the relationship between family collective efficacy and open communication. International Journal of Environmental Research and Public Health 16(24): 5006.

66.

Ritchwood

Malo

Jones

, et al. (2020) Healthcare retention and clinical outcomes among adolescents living with HIV after transition from pediatric to adult care: A systematic review. BMC Public Health 20(1): 1195–1195.

67.

Sánchez

Ramay

Zook

, et al. (2021) Toward improved adherence: A text message intervention in an human immunodeficiency virus pediatric clinic in Guatemala City. Medicine (Baltimore) 100(10): e24867–e24867.

68.

Senap

N. M. V.

Ibrahim

(2019). A review of heuristics evaluation component for mobile educational games. Procedia Computer Science, 161, 1028–1035,

69.

Shenderovich

Boyes

Esposti

, et al. (2021) Relationships with caregivers and mental health outcomes among adolescents living with HIV: A prospective cohort study in South Africa. BMC public Health 21(1): 172.

70.

Sherr

Hensels

Tomlinson

, et al. (2018) Cognitive and physical development in HIV-positive children in South Africa and Malawi: A community-based follow-up comparison study. Child: Care, Health & Development 44(1): 89–98.

71.

Smith

Rafferty

(2020) The implementation research logic model: A method for planning, executing, reporting, and synthesizing implementation projects. Implementation Science: IS 15(1): 84–84.

72.

Stankievich

Malanca

Foradori

, et al. (2018) Utility of Mobile communication devices as a tool to improve adherence to antiretroviral treatment in HIV-infected children and young adults in Argentina. Pediatric Infectious Disease Journal 37(4): 345–348.

73.

Stoumpos

Kitsios

Talias

(2023) Digital transformation in healthcare: Technology acceptance and its applications. International Journal of Environmental Research and Public Health 20(4): 3407.

74.

Sukhera

(2022) Narrative reviews: Flexible, rigorous, and practical. Journal of Graduate Medical Education 14(4): 414–417.

75.

Taba

Allen

Caldwell

PHY

, et al. (2022) Adolescents’ self-efficacy and digital health literacy: A cross-sectional mixed methods study. BMC Public Health 22(1): 1223.

76.

Tarantino

Lartey

Arnold

, et al. (2022) Preferences for a game-based SMS adherence intervention among young people living with HIV in Ghana: A qualitative study. AIDS and Behavior 26(7): 2169–2181.

77.

Vanden Abeele

MMP

Nguyen

(2022) Digital well-being in an age of mobile connectivity: An introduction to the special issue. Mobile Media & Communication 10(2): 174–189.

78.

Van Wyk

Kriel

Mukumbang

(2020) Retention in care for adolescents who were newly initiated on antiretroviral therapy in the Cape Metropole in South Africa. S Afr J HIV Med 21(1): a1077.

79.

WHO (2016) Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Available at: https://apps.who.int/iris/bitstream/handle/10665/252183/?sequence=1 (accessed 9 May).

80.

WHO (2018) Classification of digital health interventions. Report no. Report Number|, Date. Place Published|: Institution|.

81.

WHO (2020a) Youth-centred digital health interventions: a framework for planning, developing and implementing solutions with and for young people. Report no. Report Number|, Date. Place Published|: Institution|.

82.

WHO (2020b). Digital implementation investment guide (DIIG): integrating digital interventions into health programmes. World Health Organization. https://iris.who.int/handle/10665/334306. License: CC BY-NC-SA 3.0 IGO

83.

Youth Cultural Development (2023) YCD Toolkit. Available at: https://www.ycdtoolkit.com/0-using-the-toolkit (accessed 9 May).

84.

Zanoni

Sibaya

Cairns

, et al. (2019) Barriers to retention in care are overcome by adolescent-friendly services for adolescents living with HIV in South Africa: A qualitative analysis. AIDS and Behavior 23(4): 957–965.