Abstract
Background
Voluntary medical male circumcision (MC) is a biomedical HIV prevention method that requires post-operative follow-up for healing confirmation. Recent research found that a two-way texting (2wT) app providing SMS-based telehealth for MC patients was safe and reduced provider workload. We evaluated 2wT usability among MC clients in South Africa assigned the 2wT intervention within a larger randomized controlled trial (RCT) of 2wT safety and workload.
Methods
This quantitative usability study is within an RCT where 547 men used 2wT to interact with an MC provider via SMS. The sub-study involved the first 100 men assigned to 2wT who completed a usability survey 14 days after surgery. Acceptability was assessed through 2wT response rates of the 547 men. Regression models analyzed associations between age, wage, location, potential adverse events (AEs), and 2wT responses.
Results
Men assigned to 2wT found it safe, comfortable, and convenient, reporting time and cost savings. High response rates (88%) to daily messages indicated acceptability. Age, wage, and location didn't affect text responses or potential AEs.
Conclusion
2wT for post-MC follow-up was highly usable and acceptable, suggesting its viability as an alternative to in-person visits. It enhanced confidence in wound self-management. This SMS-based telehealth can enhance MC care quality and be adapted to similar contexts for independent healing support, particularly for men.
Keywords
Introduction
Digital health interventions are gaining traction to improve patient engagement and clinical outcomes.1–3 Currently, there are over 100,000 health-related apps and digital health interventions. 4 The World Health Organization (WHO) reported that 112 countries have at least one digital health initiative 5 and over 50 million people worldwide use app-based self-health assessments.6,7 In many low- and middle-income countries (LMICs), the shortage of healthcare workers (HCWs) combined with overburdened health facilities, call for mobile health (mHealth) interventions to address both the HCW burden and gaps in quality care.7,8 The spread of mobile phones throughout sub-Saharan Africa affords a critical opportunity to address health inefficiencies and inequalities and improve quality of care.9,10 Previous research has demonstrated that one-way and two-way short message service (SMS) text messaging interventions can support clients with health education and are convenient for care. SMS-based interventions are also low-cost and accessible, especially to young people.11–13
Sustained engagement of males in healthcare services is a perennial challenge.14,15 Across healthcare contexts, including HIV-related interventions, men are less likely to attend appointments, adhere to instructions, or be proactive in care seeking.16–18 Despite men's poor outcomes as compared to outcomes among females, men have been largely left out of HIV prevention, treatment and care interventions, which mostly target mothers, infants, or other at-risk groups.19,20 To overcome common barriers to care, including competing priorities of work, perceptions of poor-quality services, and a shift away from clinical settings,15,18 digital health interventions show promise to improve men's engagement in health services.
Several studies on digital innovations were geared towards increasing male engagement and reducing post-operative complications.11,21–23 In Zimbabwe, a country with high mobile phone ownership, severe HCW shortages, and rapid scale-up of voluntary medical male circumcision (MC) services, a two-way texting (2wT) follow-up approach to provide MC post-operative care found that 2wT was safe as compared to routine in-person visits while also providing additional benefits of reduced healthcare worker workload, male engagement in care, and lower costs.11,21–23 Adverse events (AEs), although rare, benefited from swift identification and referral to timely management using the 2wT telehealth intervention over poorly attended post-operative visits.22,24 Men's role in monitoring healing and prompt action for potential complications contributed to improved quality of post-operative MC follow-up using 2wT over scheduled in-person reviews.
A subsequent randomised controlled trial (RCT) of the 2wT approach as compared to routine in-person reviews was conducted in rural and urban South Africa. The South African 2wT RCT determined that 2wT was safe for clients; 25 acceptable and usable by healthcare workers; 26 and lowered program costs 27 as compared to standard care (routine post-operative visits). In South Africa, the RCT found that 2wT ascertained 2.3% AEs (95% confidence interval [CI] 1.3–4.1) as compared to 1.0% AEs (95% CI 0.4–2.3) among control participants, and when AEs were found earlier, treatment could prevent severe AEs, improving client care. 25 However, to be successful at scale, 2wT must also be highly usable and acceptable for male MC clients in South Africa. Therefore, we conducted a usability and acceptability assessment among 2wT arm participants embedded within the South Africa RCT to determine levels of MC client participation, perspective, and usefulness among these clients to inform 2wT expansion.
Methods
Theory of change for MC client uptake of 2wT
The introduction of a new technology or method into healthcare practice requires change-management and active engagement with end-users to determine usability, accessibility, acceptability, and barriers to uptake. We use the Fogg behavioural model (FBM) (Figure 1) to explain how 2wT-based telehealth was designed to be both acceptable and usable for male client follow-up as an alternative to routine, in-person follow-up visits. The FBM is based on the equation that behaviour change = motivation + ability + prompt and had been successfully applied in the digital health context.28,29 The model asserts that for an individual to perform a target behaviour, they must simultaneously: (1) be motivated; (2) have the ability to perform the behaviour; and (3) be triggered by a prompt to perform the behaviour. Motivation is defined by how inclined an individual is to engage in a behaviour. Ability is defined by how easy or hard it is to engage in a behaviour. Prompts are characterised by appropriate or effective cues to action, such as easy or hard prompts. The highest likelihood of behaviour change occurs when there is medium to high motivation, the behaviour is easy to do, and the prompt to change the behaviour is easy to follow.28,29 For MC, the 2wT app was designed to target the behaviour change of active engagement in post-operative telehealth as an alternative to required, in-person, post-operative visits. In 2wT, the anticipated motivation for men to participate in 2wT is the desire for healthy post-operative recovery without having to visit the clinic. Men's ability is to use their mobile phone correctly and consistently to participate in 2wT, engaging in direct SMS telehealth with a nurse when needed or desired. 2wT men received a prompt to action via the daily SMS from the 2wT platform to the client reminding them to report any challenge, concern or potential complication in their healing process.
Fogg behavioural model (FBM).
2wT technology overview
The 2wT system is built as an SMS-based platform that leverages the community health toolkit (CHT) for conversational messaging between healthcare professionals and clients. 30 The CHT is an open-source global public good software that is highly configurable, runs offline, supports multiple languages, works on basic phones, smartphones, tablets, and computers, supports multiple hierarchies and users. The 2wT was adapted from Zimbabwe and optimised for the local South Africa context to enhance provider prompts, reflect routine work flows, provide data for patient monitoring, and reflect client messaging preferences.25–27
2wT intervention
A detailed description of the 2wT intervention and the clinical results were previously published. 25 In brief, before RCT implementation, the content for the 2wT messages was pretested for understandability and clarity. A brief pilot with 50 men enrolled onto the RCT 2wT arm further improved SMS inbox modification, message format preferences and optimal message delivery (timing, frequency, language preferences). For the RCT, males receiving MC services were randomised in a 1:1 ratio and divided equally into urban and rural participants. Across all settings, the comparison was between 2 groups of adult participants, ages 18 and older, with cell phones: (1) participants who received routine care via scheduled in-person follow-up visits on days 2 and 7 (routine standard of care (control)) and (2) participants who received and responded to a daily SMS text message (2wT intervention). Men randomised to 2wT selected their preferred language for the automated daily SMS at enrollment (English, Isizulu, or Setswana); males could respond to daily messages in any language. A daily automated message was sent at 8 am, requesting a 1-digit response (0/1): 0 = no issues and 1 = experiencing a potential complication. Men who responded with a concern received an automated follow-on message asking the client to identify their specific concern. Thereafter, all 2wT exchange was direct client-to-provider communication in their preferred language to explain symptoms, receive counseling or wound care instructions, receive reassurance, or be referred by the nurse for in-person care. Follow-up communication could be initiated by the client or the nurse in any language and at any time through 2wT. Clients or nurse could make calls as requested or needed; client requests for calls were uncommon. Providers typically responded during routine clinic hours. Men were encouraged to return to the clinic if referred or if they desired. Men in both arms were asked to return on post-operative day 14 for a study-specific review and were compensated with a mobile phone airtime credit of US $7 (100 Rands) upon completion of the study visit.
Usability study design
The parent RCT and this sub-study were conducted in 3 MC facilities in the urban Ekurhuleni District (Gauteng Province), and 4 rural MC outreach sites in the Bojanala District (North West Province). We assessed the acceptability of the 2wT system using response data from all 547 2wT participants in the 2wT database. Acceptability reflects how clients interact with a system, and can be measured by frequency, consistency, and correct use of the system to communicate as intended. 31 We assessed the usability of the 2wT system from the client perspective using a self-administered questionnaire. Usability is defined as the ability to use the product to complete a task effectively, with minimal errors, efficiently and with satisfaction by users in a specified setting. 32 For usability, we focused on the usefulness of the system and user perceptions of the importance of system functions. We hypothesised that 2wT acceptability and usability would be high in these routine RCT settings, aiding program replicability and scalability and supporting a new standard of post-operative telehealth across the region. Feedback from usability responses were intended to inform 2wT improvements.
Usability study participants
RCT participants, including the 547 in the 2wT arm, were described in detail previously. 25 For the usability sub-set, the first 100 2wT men who volunteered to respond to an additional questionnaire on the day 14 study visit were purposefully selected from all 547 2wT arm males, aiming for equal representation from urban and rural participants. For acceptability, 2wT data from all 547 men assigned to 2wT were included. Table 1 details the study activities by arm. Participants received no additional compensation for completion of the usability study.
Characteristics of 2wT arm men.
Age not recorded for one participant.
Percentages are rounded up.
Participant eligibility
Participants in the sub-study were drawn only from the 2wT arm. Eligibility for the parent RCT study included: (1) 18 years old or older; (2) have access to a cell phone at the time of enrollment; (3) provide relevant contact details (phone number, address, next of kin); (4) had surgical MC and no intraoperative AE on the day of enrollment; (5) were willing to follow national MC protocols; (6) were willing to come to the clinic on day 14 post-MC; (7) were willing to respond to a questionnaire administered via SMS 21 days after circumcision; (8) provided a written informed consent; and (9) received a text confirming study enrollment.
Recruitment and consent
Urban recruitment occurred in the Gauteng province at 3 static MC sites in the Ekurhuleni district (Tembisa, Tsakane, and Katlehong North). Rural recruitment was conducted in the Bojanala district across outreach sites (Mogwase, Bafokeng, Bapong and Letlhabile). Participants in the RCT were consented for the optional usability assessment as part of overall RCT enrollment. At the study-specific day 14 visit, 2wT usability participants were recruited from rural and urban sites by a 2wT study coordinator and asked to participate in the brief, self-administered usability assessment.
Data collection and management
Acceptability
Enrollment data including enrollment site, age, and wage were extracted from the 2wT platform. Age was categorised into 18–24, 25–34, 35–49 and >50. Daily wage was categorised into None; greater than 0 to $11; $12-$19; and >$20, (at the exchange rate of US $1 = 15.68 Rands) (June 19, 2022) and site location was categorised into urban versus rural.
Usability
A brief self-administered questionnaire was completed on day 14 by the first 100 men enrolled into the 2wT arm who agreed to participate; 18% (100/547) of the RCT 2wT arm were included in the usability assessment. To assess themes like usability, usefulness, satisfaction, a 5-point Likert scale (1 = strongly agree; 5 strongly disagree) was used.
Statistical analysis
Descriptive statistics were used to summarise the baseline characteristics of the study participants as well as to present the SMS response intensity and potential AEs (PAE) response. In this study, SMS response intensity was defined as the proportion of daily SMS responded to by the participant out of a total of 13 daily potential responses. A participant PAE is defined as a response to any daily 2wT message with a ‘1’ or yes – affirming a potential complication or concern on that day. We examined associations between SMS response intensity with participants characteristics (age, wage and location) using a generalised linear model of a binomial distribution with logit link function. To determine if some men were more likely to report PAEs, and therefore target some users more effectively, the association between PAE responses (Yes [‘1’] or No [‘0’]) and participants characteristics was assessed using logistic regression. Results were reported as odd ratios (OR) with 95% CIs and level of significance of 5%. Only significant variables were included in the multivariable analyses for the two models. A 5-point Likert scale data for usability assessment was summarised using means and standard deviations (SD).
Ethics
This usability and acceptability study is embedded in an RCT registered at ClinicalTrials.gov (ID: NCT04327271) and approved by the Internal Review Boards of the University of Washington (Study 00009703, PI: Feldacker) and the University of the Witwatersrand, Human Research Ethics Committee (Ethics Reference No: 200204, PI: Setswe). Written informed consent for the usability assessment was obtained at RCT enrollment.
Results
Baseline characteristics of the 2wT participants
Enrollment for the parent RCT study started June 07, 2021 and ended February 03, 2022. In this sub-study of 100, recruitment started June 22, 2021 and ended on September 17, 2022. The majority of participants in this usability assessment were between the ages of 25 and 34 (36%), followed by those aged 35 and 49 (33%) and 18 and 24 (27%) (Table 1). Only a small percentage (4%) was 50 years old or older. Half of the participants lived in urban areas, while the other half lived in rural areas. The majority of participants (78%) selected English as the language for their automated daily message, with smaller percentages selecting Setswana (9%) or isiZulu (14%). Vodacom was the most popular cell phone company among participants (42%), followed by MTN (39%) and Telkom (11%). Slightly less than half of the participants reported receiving a daily wage (42%), with the majority earning between $12 and $19 per day (40%) and 18% earning less than $12 per day.
2wT acceptability
Men responded consistently to their daily texts across the 13 days with daily response rates ranging from 74% to 85% (Figure 2). PAE responses were lowest on day 1 (5%) and peaked on day 3 (11%). Reported PAEs remained below 10% on days 5 through 13, with only 6% reported issues by day 13.
Response rates on 2wT system over time, by response type (N = 547).
SMS response intensity and potential AEs response
Of the 547 participants, 33 (6%) participants did not respond to any messages. The median response intensity was 92%, with an interquartile range 77% to 100%. During the follow-up period, 295 (54%) reported a PAE at least once, with the number of PAE SMS responses ranging from 1 to 6 per participant. Univariable analysis showed no statistically significant associations between age, wage, and MC site location and SMS response intensity (Table 2). Similarly, the number of PAE responses was not significantly associated with age, daily wage and MC site location.
Factors associated with texting response rate and potential AE texts.
2wT usability
Responses to the self-administered questionnaire were indicative that most clients understood the 2wT process (63%), felt comfortable (98%), felt safe (92%) and would recommend this approach to their friends (87%) (Table 3). The majority of clients (60%) indicated that 2wT saved them time and money from attending in-person clinic visits. Only 20% of the men felt that the daily text was a burden, with 22% reporting challenges with sending texts and 17% reporting challenges with receiving texts.
Responses on 2wT acceptability and usability from 2wT clients.
1, Strongly agree; 2, Agree; 3, No opinion; 4, Disagree; 5, Strongly Disagree.
Discussion
Results from 2wT acceptability and usability among 2wT clients in South Africa indicate high acceptability and usability of the 2wT telehealth approach for post-MC follow-up, confirming previous findings.23,33,34 Clients reported confidence in 2wT, receiving reassurance and clinical support, but without the added time or money incurred from an in-person review. Age, wage, and location of the clients did not affect the high overall SMS response rate, suggesting that the intervention was received favourably across all patients. The usability study in combination with the RCT clinical outcomes 25 demonstrates that expectation for healthcare provider-driven follow-up care can be reduced in support of facilitating increased male self-reliance for uncomplicated MC care. We present several take-away messages from these results.
First, accessible telehealth approaches, like 2wT, appear highly usable and acceptable for men. 23 Men are typically reluctant to access healthcare in clinical settings, partially due to work or high mobility, resulting in late treatment seeking and advanced disease.16–19 However, 2wT provided remote clinical oversight for wound monitoring, offering a safe alternative for mobile or migratory men who may disengage with typical static site service delivery. Men engaged in 2wT similarly across age, location, and wage categories, with 88% responding over the 13-day follow-up, 25 suggesting that 2wT was an equitable approach to care. Of particular interest, is that older men seemed to have engaged more than their younger counterparts, contradicting initial concerns that age could present a barrier for digital health uptake. This finding is similar to the usability and acceptability study conducted in Zimbabwe.22,23 These findings are supportive of other similar studies, which found that 2wT telehealth approach for MC follow-up among adult men appears highly usable and acceptable.23,33,34 Men engaged in the system reported confidence and satisfaction in using this approach and reported preference to this mode of care above returning to the clinic for an in-person review. The 2wT intervention enabled men to monitor and self-manage their healing process, with encouragement to seek early care for PAEs. In public health programs where care or follow-up monitoring interventions can be placed within the realm of the client, this hybrid automated-human interaction platform may usher in a new state of task-shifting from formalised healthcare to client self-reliance. The open-source 2wT platform enables replication across diverse care settings, aiding scale-up and widespread adoption.
Moving away from routine settings where there is an over-reliance on the medical system, 2wT provides opportunities to decongest clinical care spaces and gives patients agency over their healthcare. 23 In the 2wT model, men's care seeking showed a preference for convenient and flexible engagement opportunities with a provider. Although the 2wT service is not framed as replacing clinical oversight entirely, it does provide a suitable and safe alternative to patient review that can reduce burden of care at clinical settings whilst serving to rapidly identify AE or sub-optimal wound-healing progress. 2wT could be used as a bridging platform to ensure that any AEs can be timeously identified and treated without unnecessary medical intervention for the majority who require no post-operative clinical reviews. Findings reinforce other studies’ findings by demonstrating that 2wT is suitable for rural contexts. 23 This could integrate MC with the cultural choices of the men who undergo traditional MC in South Africa, where men heal in community settings. Furthermore, for men who are mobile and do not necessarily remain within the area of where clinical services were provided, the 2wT system offers a safe alternative via remote clinical oversight wound monitoring.
2wT acceptance and use patterns are aligned with Fogg's behavioural model. The men were motivated to take care of their wound, able to identify any PAEs, found it easy to use 2wT, and prompted to act. This culminated in successful wound-care management and reduction in unnecessary clinic visits as identified in the clinical results. 25 These behavioural patterns were reinforced by the repeated prompts and reinforcing format of the intervention. Motivation was initially raised via enhanced post-operative counselling by trained providers and continued via daily SMS-based check-ins. Improved self-efficacy was likely a result of observed healing over the follow-up period, giving men confidence to self-manage their wounds. Daily responses reduced over time, suggesting that men who engaged via 2wT shifted from reliance on HCW guidance or reassurance towards self-reliance in monitoring their wound-healing progress.
High acceptability and use reflected intentional pre-testing and optimisation efforts to adapt 2wT for the local context. Providing the SMS service in the preferred morning hours and in local languages increased the reach of the service delivery of 2wT. In this study 57.5% of participants opted for SMS delivery in Setswana, a local language, and response were highest in the morning hours, showing that contextual factors and local relevance merit incorporation in the digital design. By allowing men to respond at any time and choose the language that was most comfortable for them, response rates were high. The provision of the service in the participant's preferred language also likely serves to increase participation and satisfaction, increasing engagement in the system and in care.
Where other mHealth interventions fall short of desired engagement with clients, 2wT addresses several shortcomings. First, 2wT is telehealth where the patients engage with a real MC-trained clinician and not a bot. After the initial automated communication for efficiency, all follow-up interaction is direct client-to-provider communication. Clients understand and appreciate personalised care. Second, 2wT facilitated in-person reviews on a needs-basis, referring clients for an in-person visit if the client wanted or needed an in-person review and for confirming visit outcomes. Responses aligned to expectations associated with healing progress, with higher interaction related to bandage removals or early bleeding on day 2 and subsequent concerns over the suture line healing over time. Requests for help reduced in the latter stages of the 13 days. For clients and clinicians, interaction demonstrated improved, not reduced communication, confirming that 2wT-based care was an augmentation, and not a replacement, for in-person visits. Lastly, this system was locally adapted and refined, fitting the intervention to the context through an iterative process with routine service providers. This optimisation likely ‘right-sized’ 2wT, helping ensure that both patients’ and providers’ considerations were incorporated into the 2wT specifications.
Limitations
First, the participants of this study represented men with access to cell phones and sufficient network coverage as that was an eligibility criteria of the parent RCT. In resource-constrained settings, phone ownership is not universal, and 2wT limitations may be compounded by low connectivity in rural areas where cell phone coverage may be lacking. This could impede the effective scale-up of the system and reduce accessibility to clients who will likely benefit the most from this system. Second, WhatsApp, as a ubiquitous messaging service preferred by many over SMS, is not currently a feature supported by the 2wT platform. Future scale-up should consider adding WhatsApp capability. 2wT was provided free of cost via SMS. However, it is possible that should SMS costs be borne by the South African MC patients, response rates could potentially decrease. Third, males who were illiterate in the common languages of the RCT implementation areas (English, isiZulu or Setswana) could neither consent nor participate, which reduced generalisability and indicates an improvement for scale-up.
Conclusion
The 2wT telehealth approach for MC follow-up was highly usable and acceptable in South Africa, giving men agency over their healthcare. Men appreciated the convenient and flexible engagement with a MC provider and demonstrated that they responded well to educational prompts with early care seeking for PAEs. Although the 2wT service is not framed as replacing clinical oversight, as demonstrated in the RCT results, 25 it does provide a suitable and safe alternative to patient review that can decongest clinical care spaces whilst serving to rapidly identify AE or sub-optimal wound-healing progress. In public health programs where care or follow-up monitoring interventions can be placed within the realm of the client, an automated-human interaction platform may facilitate task-shifting from formalised healthcare to client self-reliance. 2wT could be used as a bridging platform to ensure that any AEs can be timeously identified and treated without unnecessary medical intervention for the majority who heal independently. The open-source 2wT platform enables replication across diverse care settings, aiding scale-up and widespread adoption.
Footnotes
Acknowledgements
The authors would like to thank the Departments of Health of the Gauteng and North West Provinces; the implementing partner, Right to Care, for enabling participation of the clinical teams; and the CHAPS study implementation team for their dedication and skill in study recruitment, data management, and data capture. The authors would like to thank the Medic team, specifically Dr Raymond Mugwanya, for their technical skill and invaluable commitment to the success of the project.
Contributorship
CF, GS and JCP conceptualised the research and CF conceived the study. CF, GS and JCP provided oversight and technical support in the implementation of the study, FN, CK and JCP provided daily implementation support of the study. FO, BW and EW developed the interactive texting system and supported the daily management of the digital technology. SD, VN, HT and YS supported the data analysis for the study. All authors contributed to the writing and revisions of the manuscript.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Institute of Nursing Research (NINR) of the National Institutes of Health under award number 5R01NR019229, ‘Expanding and Scaling Two-way Texting to Reduce Unnecessary Follow-Up and Improve Adverse Event Identification Among Voluntary Medical Male Circumcision Clients in the Republic of South Africa’. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Guarantor
CF and JCP.
