Abstract
Objective
This study aimed to compare the effectiveness of instant versus text messaging intervention (TMI) on antiretroviral therapy (ART) adherence among men who have sex with men (MSM) living with HIV.
Methods
This study was conducted in an infectious disease hospital of Jinan, China from October 2020 to June 2021, using non-randomized concurrent controlled design to compare the effectiveness of instant messaging intervention (IMI) versus TMI. The intervention strategies (health messaging, medication reminder, and peer education) and contents were consistent between the two groups, and the difference was service delivery method and type of information. The primary outcome was the proportion of achieving optimal ART adherence, defined as never missing any doses and delayed any doses more than 1 hour.
Results
A total of 217 participants (including 72 in TMI group and 145 in IMI group) were included in the study. The proportion of achieving optimal adherence was higher in IMI group than TMI group at the first follow-up (90.2% versus 77.6%, p = 0.021) and second follow-up (86.5% versus 76.6%, p = 0.083). The effect of IMI versus TMI on improving ART adherence was found not to be statistically significant (risk ratio (RR) = 1.93, 95% confidence interval (CI): 0.95–3.94) in complete-case analysis. However, when excluding participants who did not adhere to the interventions, a significant improvement was observed (RR = 2.77, 95%CI: 1.21–6.38). More participants in IMI group expressed highly rated satisfaction to the intervention services than those in TMI group (67.3% versus 50.0%).
Conclusions
The IMI demonstrated superior efficacy over TMI in improving ART adherence and satisfaction with intervention services. It is suggested that future digital health interventions targeting ART adherence should prioritize instant messaging with multimedia information in areas with Internet access.
Trial registration
The study was registered at the Chinese Clinical Trial Register (ChiCTR), with number [ChiCTR2000041282].
Keywords
Introduction
High-level adherence to antiretroviral therapy (ART) is critical for viral suppression, which can not only improve individuals’ health outcomes, but also prevent onward HIV transmission. 1 Multiple barriers (e.g. social stigma, discrimination, unstable housing) impeded men who have sex with men (MSM) living with HIV to access to high-quality HIV care and decreased the chances of achieving optimal ART adherence.2,3 Globally, suboptimal adherence among MSM living with HIV has been reported in both high-income countries (e.g. US) 3 and low- and middle-income countries (e.g. Kenya, India).4,5 In China, a meta-analysis showed that MSM were more likely to report suboptimal ART adherence compared with heterosexual transmission group (coefficient = 0.36, 95% confidence interval (CI): 0.01–0.72). 6 A cross-sectional study found that about one quarter of MSM living with HIV reported non-adherence to ART dosing schedule. 7
Digital health interventions, including non-Internet technologies (e.g. text message/short message services (SMS)) and Internet-based technologies (e.g. instant message), can provide highly accessible and adaptable platforms to promote ART adherence. 8 As early as 2014, there were nearly as many mobile phone subscriptions as there were people globally. 9 Due to the high accessibility to mobile phone, text messaging interventions (TMIs) have been widely used to improve ART adherence.10,11 The World Health Organization (WHO) guidelines recommended to include text messaging for improving ART adherence as component of a package of adherence interventions. 12 However, recent literatures showed that the effectiveness of TMI on ART adherence was mixed,13,14 potentially influenced by the intervention characteristics (e.g. delivery frequency, whether the messaging is two-way, and whether the interventions include personalized contents) and participants’ background characteristics (e.g. age and ART duration). Simultaneously, it was noteworthy that the rates of personal text messaging have decreased in recent years following the availability of smartphone and instant messaging platforms. 15
With the rapid advancements in technology and increased availability to the Internet, instant messages have exploded in popularity and become an integral part of daily life. The popular instant message platforms, WhatsApp and Facebook Messenger, have had over two billion and 931 million monthly active users, respectively, as of January 2023. 16 WeChat, the most popular instant message platform in China, had over 1.3 billion monthly active users globally. 16 Compared with text messaging, instant messaging platforms offer a more interactive and social experience, which allow users to see when their friends were online, engage in group chats, share photos and videos, and use emojis and stickers to express emotions. 17 Additionally, instant message platforms often have features like status updates and personal profiles that allow users to customize their online presence and connect with others on a deeper level. The versatile features of instant messaging can contribute to form social ties, access health information, obtain emotional support, and improve health outcomes. 18 However, to our knowledge, instant messaging has not yet been fully explored in support of ART adherence, especially in low- and middle-income countries. Some instant messaging interventions (IMIs) are still in development or preliminarily testing.19,20 Furthermore, there are few innovative adherence interventions that specifically target MSM living with HIV, a marginalized community.
Therefore, in the changing technological landscape, it is prudent to reassess the effectiveness of text messaging and explore the possibilities of incorporating multimedia information in instant messaging for intervention purposes. This study aimed to compare the effectiveness of IMI versus TMI on ART adherence among MSM living with HIV in China, using cohort data from a randomized controlled trial (RCT) evaluating the effectiveness of a differentiated digital intervention on ART adherence. 21 The parent RCT consisted of two parallel arms (intervention arm and control arm), each including three groups (text message, only instant message [WeChat], or instant message plus social media [WeChat plus QQ]), with participants assigned to groups according to their preferences. Based on the consistent intervention strategies and contents, this study focused on assessing the effect of different service delivery methods (instant messaging versus text messaging) and types of information (multimedia contents versus text contents) on ART adherence.
Methods
Study design
The study adopted non-randomized concurrent controlled design to compare the effectiveness of IMI versus TMI on ART adherence in Jinan, Shandong Province of China from October 2020 to June 2021. Jinan is the capital of Shandong Province and is one of the cities which has the highest prevalence of HIV in Shandong Province. Male-to-male transmission is the main transmission route of HIV in Shandong, accounting for 69.3% of new infections in 2020. 22
In this secondary analysis, we only used cohort data from two subgroups (i.e. TMI group and IMI group) within the intervention arm of the parent RCT, consisting of a baseline survey and two follow-ups with 3-month interval (Figure 1).

Flowchart of study participants.
Participant recruitment and follow-up
All participants were recruited from an infectious disease hospital from October to December in 2020 in Jinan. The hospital provided free ART services for more than 1,600 MSM living with HIV. To be eligible, men should be 18 years or above, live with HIV and currently receive ART in the hospital, have anal sex with men at least once during their lifetime, and be willing to provide phone number or WeChat account to receive intervention materials. Besides, they were also asked to provide their treatment IDs (assigned by the hospital) for follow-up. Men were excluded if they did not have their own mobile phones (for TMI/IMI group) or could not access Internet (for IMI group), or could not complete the survey due to psychiatric disorders. The participants were recruited when they came to refill ART medications. First, the trained health providers screened the eligibility of the patients and referred eligible ones to a waiting room. Then, the trained investigators in the waiting room further identified their eligibility and informed them about the study aim, procedure, potential benefits and risks, and the voluntary and confidential nature of the study. Those who showed interests and met eligibility criteria were invited to participate in this study.
After providing written informed consents, all eligible participants needed to complete a self-administered questionnaire survey at baseline and then every 3 months, for a total of three times over a 6-month period. Men were reimbursed about 7.85 USD for participating in each survey if their questionnaires passed the quality check.
Interventions
As displayed in Figure 2, the intervention strategies (including health messaging, medication reminder, and peer education) and contents were consistent between the two intervention groups, and the difference was the service delivery method (instant messaging versus text messaging) and type of information (multimedia contents versus text contents). For health messaging component, we delivered ART medication messages biweekly and HIV clinical messages monthly using text in TMI group and using images in IMI group. For medication reminder component, we developed a SMS platform and WeChat mini program in TMI group and IMI group, respectively, to provide daily medication reminders to those who needed the service at 3 min before their dose-timing. For peer education component, after the first follow-up, we delivered peer education stories biweekly using text in TMI group and using videos in IMI group. The images and videos (Available from: https://github.com/JKD-epidemiology/Intervention-materials.git) were designed by the cooperated MSM organization and completely consistent with the text contents.

Intervention profile. Abbreviations: TMI: text messaging intervention; IMI: instant messaging intervention; SMS: short message service.
Variables and measurement
At baseline survey, we collected socio-demographic information (e.g. age, education level, annual income, marital status, and sexual orientation), behavioral information (e.g. alcohol use, sexual orientation disclosure, and HIV status disclosure), and ART-related information (e.g. ART duration, frequency of medication-taking, ART regime, and experience of side effects).
The primary outcome was the proportion of self-reported optimal ART adherence, which defined as never missing any doses (percent adherence) and delayed any doses more than 1 hour (dose-timing adherence) within 1 month period. We assessed ART adherence at baseline and each follow-up. Percent adherence was measured by a question “how many times have you missed taking medication in the past month.” This self-reported measurement has been widely used10,20,23 and found to be strongly associated with HIV viral load suppression. 24 The cut-off value of optimal adherence usually varied in different studies. Although previous evidence demonstrated that a high adherence level of 95% is enough to achieve consistent viral load suppression, we adopted a more conservative cut-off value (100%) to define optimal percent adherence considering that self-report may overestimate adherence. Recent modeling evidence indicated that the importance of 100% adherence level, which found that every 10% decrease in ART adherence would need 5.07% (3.62%) increase in coverage to avoid an increase in HIV infections (deaths). 25 Dose-timing adherence was measured by a question “how often did you take medications within 1 hour of prescribed dose-timing in the past month,” with options including “never,” “occasionally,” “nearly half the time,” “more than half the time (most of the time),” and “all within the prescribed time.” The 1-hour window of dose-timing adherence was based on clinical suggestion and a digital intervention in China. 26
Secondary outcomes included CD4 T-cell counts, HIV treatment adherence self-efficacy, and quality of life. Since the hospital provided free CD4 T-cell testing for all patients from March to June every year, we collected data of CD4 T-cell counts in 2020 from electronic medical records at baseline (as pre-intervention measurement) and data in 2021 at the second follow-up (as post-intervention measurement). We used Chinese Version HIV Treatment Adherence Self-Efficacy Scale (HIV-ASES) to measure HIV treatment adherence self-efficacy at baseline and each follow-up survey. 27 The scale includes 12 items measuring two dimensions (integration and perseverance). Respondents rated on an 11-point Likert scale about how confident they were at performing behavior-related ART treatment (from 0 = “not at all confident” to 10 = “totally confident”). Scores of HIV-ASES range from 0 to 120, with higher score indicating better levels of self-efficacy. We used the World Health Organization Quality of Life HIV short version (WHOQOL-HIV BREF) to measure quality of life at baseline and each follow-up survey. 28 The scale consists of 31 items covering six domains (physical, psychological, level of independence, social relationships, environment, and beliefs), and the scores can be calculated within each domain (ranging from 4 to 20) or in total (ranging from 24 to 100), with higher score representing a higher level of quality of life.
At the second follow-up, we inquired with the participants to determine if they had seen the intervention materials or involved in the intervention activities to assess their adherence to the interventions. For those who adhered, we inquired about their satisfaction with the intervention services. The participants were asked to rate their satisfaction on a scale of 1 (unsatisfied), 2 (somewhat satisfied), or 3 (highly satisfied) in regard to the following aspects: increased knowledge of ART, establishment of healthy medication habits, boosted confidence in managing their condition, and overall satisfaction.
Statistical analysis
First, descriptive analysis was used to summarize background information and the proportions of achieving optimal ART adherence, with Chi-square tests being used to examine the statistical difference between TMI group and IMI group. Second, complete-case analysis with generalized linear mixed model (GLMM) was employed to compare the effectiveness of IMI versus TMI on primary and secondary outcomes. The GLMM was adjusted for time indicator, baseline outcome, age, education level, and ART duration. Individual participants with multiple measurements were regarded as random intercepts. Third, two sensitivity analyses were conducted. One evaluated the per-protocol effect of IMI versus TMI by excluding individuals who did not adhere to the interventions. The other sensitivity analysis used 95% adherence level to define optimal ART adherence (i.e. achieving ≥95% percent adherence and never delayed any doses more than 1 hour within 1-month period). The results of GLMMs were reported with risk ratio (RR) for categorical variables and mean difference (MD) for continuous variables, along with 95% CI. Descriptive analysis was performed using SPSS 24.0, while GLMMs were conducted using SAS 9.4.
Ethical statement
Prior to launching the study, ethical approval was obtained from Ethical Review Committee of School of Public Health in Shandong University (REC number: 20190210). The study was completed in accordance with World Medical Association Declaration of Helsinki. Written informed consents were obtained from all participants.
Results
Background information
A total of 217 participants (including 72 in TMI group and 145 in IMI group) were recruited and included in descriptive analysis. The majority were under 40 years old (78.4%), unmarried/divorced/widowed (73.7%), attended college (61.3%), and reported their sexual orientation as gay (59.0%). About half of participants (51.6%) had an annual income of USD 9,474 or less. More than half of the participants reported no alcohol use in the past 12 months (55.8%), disclosed their sexual orientation to others (57.6%), and disclosed their HIV status to others (64.5%). For ART-related information, about 60% initiated ART less than 3 years (62.7%), took ART medications once a day (57.9%), and experienced side effects during ART (61.8%). Most participants took three kinds of ART medication (91.7%). The Chi-square tests showed that there was no significant difference between the TMI group and the IMI group in terms of socio-demographic characteristics, behavioral, and ART-related information. (Table 1)
Socio-demographic, behavioral and ART-related information of the study participants (N = 217).
Missing values were not taken into account in the percentage calculation.
Abbreviations: ART: antiretroviral therapy; USD: United States Dollars; TMI: text messaging intervention; IMI: instant messaging intervention.
Has told anyone (except male partners) about sexual orientation or sexual history with men.
Has told anyone (except healthcare providers) about HIV-positive status.
Proportions of optimal ART adherence at baseline and each follow-up
At baseline, there was no significant difference for the proportions of participants achieving optimal ART adherence between IMI group and TMI group (81.4% versus 72.2%, p = 0.123). At the first follow-up, IMI group had significantly higher proportion of optimal ART adherence compared to TMI group (90.2% versus 77.6%, p = 0.021). At the second follow-up, IMI group had higher proportion of optimal ART adherence but the difference was not statistically significant (86.5% versus 76.6%, p = 0.083) (Figure 3).

Proportions of optimal ART adherence at baseline and each follow-up. Abbreviations: ART: antiretroviral therapy; IMI: instant messaging intervention; TMI: text messaging intervention.
Effectiveness of IMI versus TMI on primary and secondary outcomes
A total of 204 (94.0%) participants completed at least one follow-up and were included in GLMMs. According to the complete-case analysis, the effect of IMI versus TMI on improving ART adherence was not statistically significant (RR = 1.93, 95%CI: 0.95–3.94). However, when excluding participants who did not adhere to the interventions, a significant improvement was observed (RR = 2.77, 95%CI: 1.21–6.38). The sensitivity analysis using a 95% adherence level also observed significant improvement (RR = 2.07, 95%CI: 1.01–4.25) (Table 2). For secondary outcomes, complete-case analysis showed that there was no significant difference in CD4 T-cell counts (MD = 13.88, 95%CI: −10.40–38.15), HIV treatment adherence self-efficacy (MD = −0.16, 95%CI: −2.66–2.34), and quality of life (MD = −0.19, 95%CI: −1.76–1.38) between the IMI group and the TMI group (Supplemental material Table S1).
Risk ratio of IMI versus TMI on ART adherence among MSM living with HIV in China (N = 204).
*p < 0.05, **p < 0.01, ***p < 0.001.
Abbreviations: ART: antiretroviral therapy; TMI: text messaging intervention; IMI: instant messaging intervention.
Satisfaction toward the intervention services
At the second follow-up, 84.7% (161/190) of participants were found to be adhering to the interventions. The proportion of adherence to the intervention was 68.8% (44/64) and 92.9% (117/126) in TMI group and IMI group respectively, and the difference was statistically significant (χ2 = 19.071, p < 0.001). Among those who adhered to the interventions, 97.5% (157/161) rated their satisfactions with the intervention services. Comparatively, a greater number of participants in the IMI group, as opposed to the TMI group, reported high levels of satisfaction in the following aspects, including increased knowledge of ART (56.5% versus 47.7%), establishment of healthy medication habits (62.8% versus 52.3%), boosted confidence in managing their conditions (64.6% versus 43.2%), and overall intervention services (67.3% versus 50.0%) (Supplemental material Figure S1).
Discussion
This non-randomized concurrent controlled study compared the effectiveness of two digital strategies (innovative IMI versus traditional TMI) on ART adherence among MSM living with HIV, indicating that instant messaging may be a useful tool to optimize HIV care among the marginalized community.
Our study found that the effect of IMI versus TMI on improving ART adherence was not significant in complete-case analysis. However, when excluding participants who did not adhere to the interventions, a significant improvement was observed. The sensitivity analysis using a 95% adherence level also observed significant improvement. Besides, a greater number of participants in the IMI group, as opposed to the TMI group, reported high levels of satisfaction to the intervention services. In line with our findings, another RCT evaluating the effect of IMI versus TMI on bowel preparation quality also suggested that instant messaging had better efficacy. 29 This suggests that the service delivery method and the type of information may play a key role in the efficacy of interventions given that intervention strategies and contents were consistent between the two groups. In terms of the service delivery method, instant messaging has become the main way of inter-personal communication and information exchange. More than 90% of residents in China logged into WeChat every day and about 32% used WeChat for longer than 2 hours per day. 30 Research has shown that people living with HIV in China often prefer instant messaging for communication with friends. 20 This indicates that utilizing instant messaging for intervention delivery may enhance engagement. For the type of information, multimedia presentations (image and video) are known enhance comprehension and decision-making compared to text alone, as they require less reliance on working memory. 31 In a hypothetical trial setting, multimedia material can provide information as effectively as text material, while also improving understanding, decisional confidence, and subjective evaluations. 32 Therefore, incorporating multimedia information into IMIs for ART adherence may be beneficial in settings where Internet access is available.
Due to lack of accurate and comprehensive data on costs, we did not conduct cost-effectiveness analysis. Overall, IMI requires more upfront investment for development and maintenance, such as creating a WeChat mini program and multimedia materials, compared to TMI. However, the interactivity and personalization in IMI can lead to higher adherence and better health outcomes for PLWH. 33 This could potentially result in cost savings for healthcare systems in the long run. Additionally, IMI typically uses Internet or data connection to send messages, which can be cheaper or even free compared to traditional text messages that incur a cost per message. Further research is needed to better understand the cost-effectiveness of different digital health interventions and to inform decision-making in this area.
In this study, no significant difference in CD4 T-cell counts was observed between the IMI group and TMI group. The disconnect between self-reported adherence and biological outcomes may be due to the short follow-up duration. Past research has shown that changes in biological outcomes may be delayed following reporting suboptimal ART adherence. 34
A concern was that the proportion of achieving optimal adherence seemed to decrease over time either in IMI group or TMI group, possibly due to habituation to the interventions. In addition, we only used one-way messaging to deliver interventions in the consideration of privacy protection. Bidirectional communication can bolster the relationships between patients and health providers, which may help to achieve long-term adherence goals. 35 Incorporating interactive elements like gamification and virtual reality technology into IMI can enhance user experiences, facilitate social connections, and improve adherence. 36
Ethical considerations regarding privacy and data security are important when implementing digital health interventions. 37 Robust privacy protocols and data security measures should be in place, such as encryption strategies and regular security audits.
This study has policy and research implications. From a policy perspective, the findings would help generate evidence-based formulations to optimize HIV care and ART guidelines, suggesting healthcare providers prioritize delivering instant messages with multimedia information to improve ART adherence. From a research perspective, implementation research and process evaluations are required to test the efficacy of instant messaging in diverse real-world contexts. Moreover, it would be valuable to investigate the long-term effectiveness and sustainability of digital health interventions across different cultures, settings, and populations. The hybrid models integrating instant messaging with other forms of support could also be explored for feasibility and effectiveness.
This study is subject to several limitations. First, ART adherence was based on self-report, which may lead to social desirability bias and recall bias. However, evidence showed that there was a high agreement between self-reported adherence data and biological outcomes.23,24 Second, participants were recruited from one designated ART hospital in Jinan and most of them were young and well-educated. Generalizing the findings to other groups of MSM living with HIV and other areas should be done with caution. Third, this is a secondary analysis of a RCT and the grouping was not based on randomization. Selection bias might exist because participants were allowed to choose the digital strategies according to their preferences. Although our data showed that there was no significant difference for background information and baseline outcomes between the two intervention groups, some unmeasured confounders may interfere the interpretation of the findings. Last but not least, the relatively small sample size may weaken the power of this study.
Conclusions
In conclusion, IMI demonstrated superior efficacy over TMI in improving ART adherence and satisfaction with intervention services. This study extended the existing literatures on Internet-based IMIs and contributed to optimize HIV care. It is suggested that future digital health interventions targeting ART adherence should prioritize instant messaging with multimedia information in areas with Internet access. Furthermore, assessing the long-term effects of digital health interventions is imperative, as their effectiveness seem to wane over time.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241257447 - Supplemental material for Effectiveness of instant versus text messaging intervention on antiretroviral therapy adherence among men who have sex with men living with HIV
Supplemental material, sj-docx-1-dhj-10.1177_20552076241257447 for Effectiveness of instant versus text messaging intervention on antiretroviral therapy adherence among men who have sex with men living with HIV by Kedi Jiao, Jing Ma, Yuxi Lin, Yijun Li, Yu Yan, Chunxiao Cheng, Wenwen Jia, Jing Meng, Lina Wang, Yanwen Cao, Zhonghui Zhao, Xuan Yang, Meizhen Liao, Dianmin Kang, Chunmei Wang and Wei Ma in DIGITAL HEALTH
Supplemental Material
sj-docx-2-dhj-10.1177_20552076241257447 - Supplemental material for Effectiveness of instant versus text messaging intervention on antiretroviral therapy adherence among men who have sex with men living with HIV
Supplemental material, sj-docx-2-dhj-10.1177_20552076241257447 for Effectiveness of instant versus text messaging intervention on antiretroviral therapy adherence among men who have sex with men living with HIV by Kedi Jiao, Jing Ma, Yuxi Lin, Yijun Li, Yu Yan, Chunxiao Cheng, Wenwen Jia, Jing Meng, Lina Wang, Yanwen Cao, Zhonghui Zhao, Xuan Yang, Meizhen Liao, Dianmin Kang, Chunmei Wang and Wei Ma in DIGITAL HEALTH
Footnotes
Acknowledgements
We thank Guangmei Liu, Xiuhong Zhao, Yanmei Bi, Qian Chen, and other medical staff from Shandong Public Health Clinical Center for helping to recruit participants.
Contributorship
WM, KJ, ML, CW, and DK contributed to conception and design of the study. WM provided oversight for data collection. KJ, JM, YL, YY, CC, YL, WJ, LW, YC, ZZ, XY, and JM assisted in data collection and management. KJ analyzed the data and drafted the manuscript. WM made critical comments of the manuscript. All authors revised and approved the final manuscript. WM was responsible for the finished work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Conflicting Interests
The author(s) declare(s) that there is no conflict of interest.
Ethical approval
The Ethical Review Committee of School of Public Health in Shandong University approved the study (REC number: 20190210).
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Key Research and Development Program of China [grant number 2017YFE0103800].
Guarantor
WM
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
