Feasibility and acceptability of a dengue self-monitoring system to reduce treatment delay in Malaysia: A single-centre pilot randomised controlled trial

Abstract

Objective

Most dengue cases are managed in an outpatient setting, where patients are advised to return to the clinic daily for monitoring. Some patients can develop severe dengue at home and fail to recognise the deterioration. An application called DengueAid was designed as a self-monitoring tool for patients to reduce delay in seeking timely treatment. This study aimed to assess the feasibility of conducting a randomised controlled trial to determine the effectiveness of the DengueAid application.

Methods

Dengue patients were recruited from a public health clinic in Malaysia and randomised to either use the DengueAid application plus standard care for dengue or receive only the standard care. The outcomes evaluated were the (1) feasibility of recruitment, data collection and follow-up procedures; (2) preliminary clinical outcome measures; and (3) acceptability of DengueAid. Qualitative interviews were conducted for participants in the intervention arm to assess the acceptability of DengueAid.

Results

Thirty-seven patients were recruited with 97% (n = 36) retention rates. The recruitment rate was low (63% refusal rate, n = 62/99) with difficulty in data collection and follow-up due to the variable interval of care for dengue in an outpatient setting. DengueAid application was acceptable to the participants, but preliminary clinical outcomes and qualitative data suggested limited utility of the application. Unwell conditions of patients and limited access to healthcare are important factors impacting the application's utility.

Conclusion

The feasibility trial uncovered issues with recruitment, data collection and follow-up processes. Further research and modification to the application are needed to improve its utility and usability.

Keywords

Digital health eHealth remote patient monitoring self-monitoring digital clinical trials

Introduction

Dengue is a mosquito-borne viral disease endemic in tropical regions such as Southeast Asia, South America and central Africa with an increasing global disease burden. An estimated 100 million symptomatic dengue cases are reported annually in over 120 countries.¹ Higher dengue transmission risk is also predicted in countries not endemic for dengue by 2050 such as China, Japan, Mexico and Australia due to climate warming.² The Western Pacific region, including Southeast Asian countries such as Malaysia, accounts for 70% of the global disease burden for dengue³ with an annual increase in dengue cases, from 430,023 in 2013 to 1,050,285 in 2019.⁴ In Malaysia, from 43,346 to 130,101 laboratory-confirmed cases were reported annually from 2013 to 2019 with case-fatality ratio of 0.11%–0.28%.⁴

Dengue management can be challenging due to the risk of rapid progression to severe dengue. While most patients recover without complications, 1%–2% develop severe dengue including plasma leakage, bleeding, organ impairment, shock, and death.⁵ Patients with dengue usually present with a high fever, associated with symptoms such as headache, retroorbital pain, muscle and joint pain, nausea, vomiting, or rash.⁵ Dengue infection typically runs a self-limiting seven-day cycle which is divided into febrile, critical and recovery phases. The World Health Organization (WHO) suggests using warning signs to identify severe dengue; they are persistent vomiting, not drinking, severe abdominal pain, lethargy, bleeding (epistaxis, black stool, haematemesis, excessive menstrual bleeding or haematuria), dizziness, cold and clammy hands and feet, and reduced urine output.⁶ The absence of these warning signs is strongly correlated with non-progression of disease, signalling patients can be managed safely as an outpatients.⁷ Hence, close monitoring of warning signs with early identification and intervention before complications develop are crucial.⁵

As most dengue cases are self-limiting and treated as outpatients, doctors who manage dengue rely on patients to seek treatment in a timely manner should warning signs develop. Daily follow-up for patients remains the mainstay of treatment where doctors actively look for warning signs and changes in blood parameters such as increased haematocrit and rapid drop in platelet count.⁵ However, severe dengue can develop rapidly within 24 hours.⁵ While verbal or written self-care advice is given to patients routinely to remind them to seek help urgently if they develop warning signs, there is a wide variation in how these messages are delivered. Patients often do not fully understand and remember the health advice, especially when they feel unwell or the consultation is short.⁸ This often results in a delay in seeking help and treatment, which in turns may lead to morbidity and mortality.

One possible solution is a dengue self-management and self-monitoring digital tool to educate, guide and prompt patients to seek help appropriately. Mobile phones are increasingly being used in health care for patient education and decision-making⁹ and clinical algorithms have been incorporated into mobile phones to support patient decision-making.^10,11 Previously developed digital applications for dengue worldwide, such as Mo-Buzz and Mozzify, primarily serve as dengue surveillance and health education tools for public health.^12,13 A symptom checker function was incorporated in one dengue application but it was designed for early diagnosis of dengue, not for symptom monitoring of diagnosed dengue cases.¹²

Our team developed a mobile Dengue Self-Monitoring System (DengueAid) to allow dengue patients to self-monitor their symptoms at home, educate them about warning signs and prompt them to visit a doctor if they develop warning signs before their next scheduled follow-up. DengueAid is hypothesised to reduce delay in treatment-seeking in dengue patients. To the best of our knowledge, this is the first digital self-monitoring tool for outpatient follow-up of dengue. In this study, we conducted a single-centre pilot randomised controlled trial (RCT) to assess the feasibility of conducting a full-scale RCT to evaluate the effectiveness of DengueAid in reducing delay in treatment-seeking among patients diagnosed with dengue. Specifically, we assessed (i) the feasibility of recruitment, data collection and follow-up procedures, (ii) the preliminary clinical outcome results, and (iii) the acceptability of the dengue self-monitoring intervention using DengueAid.

Methods

Study design and setting

This is a single-centre pilot study using a mixed-method approach to collect quantitative clinical outcomes as well as qualitative data on researchers’ and patients’ views and experiences throughout the trial. A parallel 1:1 two-armed RCT was conducted and a qualitative component using in-depth interviews as well as field notes was embedded within the RCT.

The setting of this pilot trial was a public primary care clinic (also known as a health clinic) under the Ministry of Health, Malaysia. The clinic is located in an urban region with a high load of dengue cases. The patient demographic reveals a distribution of 85% Malay, 10% Indian, and 5% Chinese. Predominantly, Malay is the primary language spoken, constituting 90%, with English following at 8%, and Mandarin or Tamil at 2%. Dengue cases constitute 1%–3% of fever cases seen at the clinic. The internet penetrability rate of the patients stands at 99%. The pilot trial was conducted from 20 January 2023 to 30 June 2023 with the recruitment phase from January to April 2023. The pilot trial ended after the researchers achieved an adequate sample size and completed the data collection for follow-up.

Participant identification and eligibility

Patients with a suspected diagnosis of dengue fever were seen at a dedicated fever centre in the clinic. After the patients were seen by doctors and diagnosed as having dengue fever to be managed as an outpatient, the staff at the fever centre would refer them to the research assistant (RA), who then screened the patients for eligibility. The inclusion criteria are (1) age 18 years old and above, with (2) a clinical diagnosis of dengue fever as confirmed by medical notes, and (3) requiring outpatient follow-up. As the DengueAid application was only available for Android smartphones, patients who used iPhones or did not have a smartphone were excluded.

The RA recruited the patients either face-to-face at the fever centre or via a phone call if they had left the clinic before the RA met them. The patient read the Participant Information Sheet and was given sufficient time to consider their participation in the study. All patients who agreed to participate in the study signed an informed written consent either in hardcopy (if face-to-face approach) or softcopy (if approached via phone). Patient's sociodemographic and clinical data were collected. Only a single RA was involved in the recruitment of the patients.

Intervention and control

The intervention in this study was the DengueAid smartphone application (Figure 1), which comprised:

Symptom self-monitoring algorithm whereby patients were advised to report their symptoms thrice daily (9 a.m., 2 p.m. and 8 p.m.). If the patient reported any warning symptoms (persistent vomiting, persistent diarrhoea, abdominal pain, bleeding, difficulty breathing, feeling like fainting, difficulty in carrying out usual activities, drowsiness, reduced urine output and reduced urine output), the system would advise the patients to seek earlier medical care instead of waiting for the scheduled appointment the next day or other days.

Educational materials regarding dengue fever and self-care.

Input forms to key in important parameters such as vital signs and blood tests (haematocrit, white cell count, platelet and haemoglobin).

The questionnaire used for monitoring of symptoms in DengueAid was self-developed based on the Malaysian clinical practice guideline for management of dengue, literature reviews of dengue warning signs and existing mobile applications for disease monitoring and expert opinion.^5,11,12,14 The questions were worded in a manner that aligned with the way locals spoke and conversed. An interdisciplinary team was responsible for the development of the questionnaire, application design, overall study conception and execution. The team comprised primary care physicians (n = 4), infectious disease specialists (n = 4), emergency physicians (n = 1), computer science academicians (n = 3) and eHealth experts (n = 2). We used the Agile Software Development Life Cycle approach, incorporating a rule-based expert decision algorithm into the application.¹⁵

Figure 1.

Storyboard for DengueAid.

This application, along with the questionnaire for symptom monitoring, was available in two languages, English and Malay, to cater to the multi-ethnic population in Malaysia who may be familiar with one language over another. Malay is the national language while English is also widely used in many parts of the country. Prior to the trial, the application and the questionnaire were pilot-tested by researchers and recovered dengue patients to obtain input on the questionnaire and application design, and to ensure no errors in the algorithm and functions of the application.

Dengue patients were asked to key-in their symptoms regularly via the application when they were at home. Reminders appeared on the phone to remind them to key in the symptoms thrice daily. The other features such as educational material and input forms to key in vital signs and blood parameters were not compulsory for users. During the study period, we had a dedicated phone line to respond to any technical issues encountered by the patients, but no technical issues were raised throughout the study. The participants in the intervention arm still received the same standard care for dengue as the control arm, in addition to the DengueAid application. DengueAid was designed as an independent decision-support tool without active intervention from the researchers during their self-monitoring at home. While researchers were able to view patients’ usage of DengueAid and reports of warning signs via a digital dashboard, there were no automatic prompts sent to researchers when patients reported warning symptoms at home. To avoid influencing patients to use the application more frequently, the researchers only contacted them two weeks after their recruitment to collect outcome data.

The standard outpatient care for dengue entailed the provision of a dengue monitoring card, daily or regular appointment to monitor their blood profile and warning signs and advice to seek help immediately if they developed warning symptoms at home. In Malaysia, dengue patients who are managed as outpatients would be provided with a monitoring card which serves as a record of daily vital signs and full blood count results, and an information leaflet for home care advice and danger signs of dengue.

For the control arm, the patients received the standard outpatient care for dengue as mentioned above.

Outcome measures

This pilot trial aimed to evaluate (i) the feasibility of recruitment, data collection and follow-up procedures, (ii) the preliminary clinical outcome results, and (iii) the acceptability of the dengue self-monitoring intervention, to inform the design and delivery of the full-scale DengueAid RCT.

The clinical outcome measures of interest in this trial included percentages of (i) patients who were compliant to follow up-visits, (ii) patients who developed warning signs, (iii) patients who sought help earlier after developing warning signs, and (iv) patients with warning signs who were admitted. The proportion of patients who sought help earlier after developing warning signs was the main clinical outcome of interest. The clinical outcomes of interest were determined by discussion among researchers with input from clinicians in the Ministry of Health. The data on recruitment, which comprised recruitment methods and refusal rate, were collected continually by the research assistant (RA) using a data collection form as the trial went on until completion. Issues arising during the recruitment process were documented as field notes. The researchers conducted weekly discussions with the RA to reflect on the recruitment process and rate. The clinical outcome data were collected by RA two weeks after the recruitment of each participant with data extraction from the application dashboard (for those in the intervention arm), medical records and phone interviews. The data on the acceptability of this intervention, which comprised the application usage pattern and opinions of users, were collected from data in the application dashboard and qualitative interviews with the participants in the intervention arm.

Sample size

This pilot study was conducted to evaluate the feasibility of conducting the full RCT. The findings from the main clinical outcome in this pilot study will be used to determine the sample size of the full RCT. We followed Julious's rule of thumb to achieve at least 12 participants per study arm in this study.¹⁶

Randomisation, allocation, implementation and blinding

The patients were randomised to either using the DengueAid application on top of the standard outpatient care for dengue (intervention) or receiving just the standard outpatient care for dengue (control). We used a permuted block design for the randomisation of groups. The randomisation sequence was generated using a computer with a mixture of blocks 4 and 6 with a ratio of 1:1 between the control arm and intervention arm. It was performed by a researcher from the central office who was not involved in the recruitment of participants at the recruitment site. Block sizes were kept confidential and not shared with those involved in allocation. Allocation was informed to the RA by the researcher who generated the initial randomisation on a daily basis to ensure concealment. Finally, after the patients agreed to take part in the study, the RA then allocated them to their respective arms.

While the allocation of the participants was masked, the implementation of the intervention cannot be blinded due to the nature of the study. While participants from both arms of the study might be aware of the existence of the DengueAid application, only those in the intervention arm were allowed access to the application. DengueAid users were required to sign in with passcodes provided by the researchers after the RA registered the participants in the DengueAid dashboard. The procedure for code break was not applicable in this study due to the absence of blinding.

The follow-up of the participants was decided by the treating doctor, depending on their clinical judgement. The trial protocol did not interfere with the decision to follow-up by the treating doctors. Therefore, the interval and length of follow-up varied across participants.

After two weeks from the date of recruitment, the RA would trace the medical records and contact the participants in both arms through phone calls to collect data on their outcomes. The researchers decided on the two-week period to collect the data as they projected that the recovery from dengue would be within one to two weeks. DengueAid was designed to be used only during acute dengue illness. The RA was instructed to avoid contacting the participants within the two weeks from recruitment as there were concerns that the contact could be a form of contamination, influencing the participants’ behaviour.

Data analysis

Continuous data were checked for normality using the Shapiro–Wilk test with mean and standard deviation (SD) used for normally distributed variables. Categorical variables were reported as frequency and proportions. Recruitment and retention data were described using descriptive data and a flow diagram. The feasibility of the clinical outcome measurement and acceptability of the dengue self-monitoring intervention were described in a narrative format, supported by descriptive data. The secondary clinical outcome data were described in descriptive format. Inferential statistical analysis was not done due to the small sample size. Data from the electronic medical records, digital dashboard of DengueAid and data collection form were transferred to the Statistical Package for the Social Sciences (SPSS) version 27 software for analysis. Participants in the intervention arm who did not use the DengueAid at all were excluded from the analysis. Participants with incomplete data were also excluded from the analysis.

Qualitative interviews on the utility and usability of DengueAid intervention

The participants who used the DengueAid were approached for an interview after they completed their follow-up for dengue. The interview aimed to evaluate the acceptability of DengueAid among the participants. The interviews were conducted via the Zoom platform or a phone call. During the interview, their feedback was obtained regarding the utility and usability of DengueAid. The interview was conducted by WLN and guided by a topic guide (Supplemental Appendix 1). All the interviews were audio-recorded and transcribed verbatim. Thematic analysis was conducted where open codes were assigned by researchers to participants' views. Larger themes were created by grouping related codes together. The themes were then categorised based on the usability (a problem when installing or using the application) and utility (usefulness of the application in terms of decision making and disease understanding) of DengueAid. Data saturation was not formally assessed in this study. The interviews were conducted as part of the overall evaluation of the utility and usability of DengueAid, whereby all participants in the intervention arm who used DengueAid were invited to participate in the interviews.

Results

Recruitment, follow-up and retention

Figure 2 shows the flow of the participants during the trial. A total of 126 patients diagnosed with dengue were screened for eligibility during the pilot trial period; 27 patients did not fulfil the inclusion criteria (reasons were: patients requiring immediate referral to hospital, and did not possess an Android smartphone) and 62 participants declined to participate. We did not document the reasons for refusal during the initial phase and only started doing so in March 2023 (the second half of the recruitment period) after experiencing a high refusal rate. While most patients still did not divulge their reason for refusal, the documented reasons for refusal included lack of interest (n = 2), needing to take care of the child (n = 1), unfamiliarity with smartphone application (n = 2), family disapproval (n = 1) and language barrier in an Indian patient (n = 1).

Figure 2.

CONSORT flow of the pilot trial.

Among the remaining 37 patients who agreed to participate in the pilot trial, 18 were randomised to the intervention arm and 19 to the control arm. The retention rate at the end of the trial was 100% (18/18) for the intervention arm and 95% (18/19) for the control arm. In the intervention arm, three participants did not use the DengueAid application at all after installation. The reasons were unfamiliarity with how to use the application (n = 1) and being too unwell to use the application (n = 2). One participant in the control arm was uncontactable after recruitment and did not turn up for follow-up in the clinic.

Characteristics of study participants

Table 1 shows the characteristics of the study participants. The sociodemographic profile of participants was similar between both the intervention and control arms: mean age of 31–32 years old, predominantly male (66.7% in the intervention arm, 77.8% in the control arm) and Malay (86.7% in the intervention arm and 100% in control arm). During recruitment, the participants in the intervention arm already had symptoms of dengue for 4 days as compared to 5 days for those in the control arm (p < 0.04). The participants in the intervention arm used the DengueAid application for 3.9 days.

Table 1.

Characteristics of study participants (N = 33).

Parameter	Intervention arm (n = 15)	Control arm (n = 18)	p value
Mean age, years (SD)	31.1 (8.8)	31.8 (13.2)	0.730
Gender
Male	10 (66.7%)	14 (77.8%)	0.697
Female	5 (33.3%)	4 (22.2%)
Ethnicity
Malay	13 (86.7%)	18 (100%)
Indian	1 (6.7%)	–
Kadazan-Dusun	1 (6.7%)	–
Median day of illness upon recruitment, days (IQR)^a	4.0 (2.0)	5.0 (3.0)	0.040
Mean duration of days using application, days (SD)^b	3.9 (2.9)	Not applicable

SD: standard deviation; IQR: interquartile range, Mann-Whitney U test for age (non-normally distributed) and independent t-test for day of illness (normally distributed), Fisher's exact test for gender.

Data source from medical notes.

Data source from the application dashboard.

Feasibility of recruitment, data collection and follow-up procedures

As mentioned in the Methods, the RA initiated recruitment attempts after the patients were seen and diagnosed with dengue by doctors in the fever centre. The dengue patients were seen together with other patients with fever at the fever centre; hence there was a risk of exposure to COVID-19 infection for the RA. To comply with COVID-19 precaution protocol at the time of research, the RA was recruited via a recruitment counter outside the fever clinic. Patients diagnosed with dengue fever were instructed to approach the recruitment counter after seeing the doctor. Some patients did not turn up at the counter and hence phone call recruitment was done for these patients who had returned home. Face-to-face recruitment yielded a higher success rate of 57% (25 recruited out of 44 patients approached) compared to phone recruitment at 22% (12 recruited out of 55 patients approached).

We were able to collect the baseline demographic and clinical details such as the day of illness upon recruitment for all patients as the data were available from records and questioning of patients. For the follow-up and outcome data, the RA faced challenges to extract outcome data such as urgent health-seeking behaviour and hospitalisation solely from the medical records because some patients changed places of follow-up and the electronic medical record systems among different clinics were not linked. The interval of follow-up for each patient also differed as decided by the treating doctor. This made it difficult for RA to know when a patient would be coming again for follow-up; thus identifying compliance was also a challenge. In the DengueAid application, there was also a built-in form that would pop-up to ask whether the patients had sought help and the details of the help-seeking behaviour (time, date and place) if they developed warning signs. However, none of the patients who sought help responded to this form. During the active disease period, the RA did not call the patients to follow up to avoid influencing their application usage pattern. Out of the 37 patients recruited, the RA was unable to extract relevant outcome data for eight patients (21.6%) from the electronic medical records after two weeks of recruitment. The RA had to contact the patients to ask if they sought earlier medical care and extract relevant data such as compliance with follow-up and outcome. There may be recall bias from the patients due to this. The RA was unable to find the record for one patient who was in the control arm, nor was he able to contact the patient. Hence that patient was excluded from the final analysis. Additionally, the RA also contacted all other patients to verify the outcome data stated in the records and check for any missing outcome data. For example, a patient might have sought earlier help at another healthcare centre but did not reveal the information to the attending doctor at our study site during the follow-up.

Preliminary clinical outcome results

Table 2 reports on the clinical outcome data from our pilot trial. A total of 14 patients out of 15 (93%) in the intervention arm reported warning symptoms via the application but when asked at the end of the trial follow-up, only nine out of 15 (60%) thought they had warning symptoms. Only four out of 18 (22%) in the control arm reported warning symptoms. Only three out of the 14 (21%) patients who reported warning symptoms in the intervention arm sought earlier help, as compared to two out of four (50%) in the control arm. Four out of 14 (29%) patients with warning symptoms were eventually admitted to the intervention arm. Two were admitted when they came to seek earlier help while another two were admitted when they came for follow-up. Only one out of the four (25%) with warning symptoms in the control arm was admitted.

Table 2.

Clinical outcomes of DengueAid pilot trial.

Outcome	Intervention (n = 15)	Control (n = 18)
Patients who developed warning symptoms	14/15 (93%)	4/18 (22%)
Patients with warning symptoms who sought help earlier	3/14 (21%) ^a	2/4 (50%)
Patients with warning symptoms who were admitted	4/14 (29%)	1/4 (25%)
Patients who were compliant to follow-up visit	9/15 (56%)	13/18 (72%)

Main outcome of interest.

The compliance to follow-up visits was lower in the intervention arm (9/15, 56%) as compared to the control arm (13/18, 72%). The researchers did not specifically collect data on the reason for non-compliance but agreed that the data would be useful.

Table 3 shows the type of warning symptoms reported by the participants in the intervention arm. Most of the symptoms reported were milder, non-specific symptoms such as difficulty in carrying out usual activities (the surrogate term for severe lethargy) and drowsiness.

Table 3.

Warning symptoms reported by participants in the DengueAid application.

Symptoms	Number of times symptoms were reported in the application	Percentage (%)
Difficulty in carrying out usual activities	48	33.3
Drowsiness	40	27.8
Stomach pain	14	9.7
Dizziness	13	9.0
Persistent diarrhoea	9	6.3
Reduced drinking	7	4.9
Reduced urination	5	3.5
Shortness of breath	4	2.8
Persistent vomiting	3	2.1
Bleeding	1	0.7

Acceptability of the DengueAid application among patients

Application usage pattern

In our pilot trial, out of the 18 patients who were allocated to the intervention arm, three of them did not use the DengueAid application at all after installation. Two of them cited that they were feeling unwell to use the application while another participant said she did not know how to use the application. Of the remaining 15 participants, only one participant used the application daily until he completed his follow-up for dengue.

Regarding the requirement to report the symptoms thrice daily, seven out of 15 participants (46.7%) in the intervention arm completed three times of self-reporting daily (while they were still using the application). When we analysed which session was commonly skipped by the participants, the night-time reporting was most commonly skipped (46.1%), followed by afternoon reporting (38.5%) and morning (15.4%).

Qualitative interview on the acceptability of the DengueAid application

We also conducted qualitative interviews to explore the utility and usability of the DengueAid application. Twelve out of 15 participants in the intervention arm agreed to an interview after they had recovered and completed follow-up for dengue. Out of the remaining three participants who did not participate, two refused due to busy schedules while another defaulted the interview appointment after agreeing initially. The patient who defaulted on the interview appointment did not respond to further calls to reschedule another interview.

Generally, the participants found the application useful to help them monitor their symptoms and provide information for self-care at home. The application also gave them an idea if their symptoms were severe or not.

Utility

We discovered four themes for the utility of the DengueAid application (quotes in Table 4).

Influence of health status on application utility. The patient's health status would influence their utility of the application. While our patients appreciated the help of DengueAid to help them decide if their symptoms were severe enough to seek help earlier, they would still make their own judgement before taking action to seek help earlier. They would decide against seeking help earlier if they felt well. On the other hand, if the patients felt unwell, they would tend to skip entering their symptoms.

Access to healthcare. Access to the nearest healthcare facilities would also influence our participants’ decision to seek earlier help. Access refers to the distance of the nearest healthcare facilities, the availability of the healthcare facilities at odd hours and the availability of someone to send them to the hospital. If the warning symptoms alert appeared at night, the participants would be less likely to seek earlier treatment, mainly driven by the unavailability of operating outpatient clinics at night and non-preference to visit the emergency department. The long-distance of the clinic from home, coupled with the taxi fare, were also reasons the patient decided not to seek earlier treatment despite advice from the application.

Accuracy and comprehensiveness of the application content. Our patients were still sceptical of the accuracy of the symptom monitoring algorithm and hesitated to follow the advice. Some patients suggested a more comprehensive questionnaire to evaluate their symptoms further and delineate the severity of their symptoms.

Misunderstanding of DengueAid application as a diagnostic tool. Some participants thought that the symptom monitoring tool in the application could be used as a diagnostic tool for dengue in the future if they developed similar symptoms again. Patient 18 mentioned she would use this tool to decide whether she needs to seek treatment in the future if she has similar symptoms.

Table 4.

The themes and supporting interview quotes (based on utility and usability).

Theme	Quotes
Utility
Theme 1: Influence of health status on application utility Theme 2: Access to healthcare	(When asked why did not seek earlier help) ‘I felt I don’t have any problem … I felt healthy … so, ok I just continue monitoring at home. I also took some crabmeat soup’. (Patient 16, 40 years old, male) ‘When I received it (the advice to seek help earlier), I felt I was getting better, I felt well already. So I didn’t go’. (Patient 5, 32 years old, male) ‘I will do my own judgement, because I am not vomiting, and because I had gone to clinic already this morning, it will be the same if I go again in the afternoon’. (Patient 4, 35 years old, male) ‘Just that sometimes we feel unwell, so we cannot update our symptoms on time. And sometimes we will forget’. (Patient 4, 35 years old, male) ‘For me, three times a day (to key in symptoms) are too many, because if I am unwell, I will not be looking at my phone at all. So I won’t feel like filling in the symptoms when you are sick’. (Patient 5, 32 years old, male) ‘We also consider other factors such as … I am going alone, I don’t have energy to go, there's no friend to bring me (to the hospital)’. (Patient 5, 32 years old, male) ‘Usually at night, the clinic is not open, and we have to go to emergency department. If we go to emergency department at night, they will ask us to go back instead’. (Patient 4, 35 years old, male) ‘Hmm … I’ll consider first … because I need to go by taxi, it's difficult to go to clinic if so. Moreover the clinic is far from my home, so I have to consider the taxi fee as well’. (Patient 18, 21 years old, female)
Theme 3: Accuracy and comprehensiveness of the application content	‘I was thinking is the advice (to seek early help) correct? Because it's my first time using this application. I have my suspicion haha, let me think about it first’. (Patient 5, 32 years old, male) ‘Based on my opinion, the questions need to be more to assess my symptoms’. (Patient 3, 35 years old, male) ‘It will be useful if there’re additional questions to follow-up on existing questions, to see if our conditions are really that severe. That’ll be the best’. (Patient 16, 40 years old, male)
Theme 4: Misunderstanding of DengueAid application as diagnostic tool	(When asked if she will consider using the app again in future) ‘I will use this app first, to fill in the symptoms to see if I really have dengue, is it severe, to decide if I need to see doctor’. (Patient 18, 21 years old, female)
Usability
Theme 1: Assistance with installation helped ensure application usage	‘To download and install the app, I was helped by the assistant, I think no problem there. If he were to tell me where to find the app (in Google PlayStore) and install it myself, I would find it difficult because at that time I was feeling slightly dizzy, not so well. If I were to do it myself at home, I think I’ll be lazy to do it, I better rest if so. So, it is better to have assistance in this’. (Patient 14, 33 years old, male)
Theme 2: A more vibrant and attractive design was desired	‘Ya, the colour (display) looks at bit pale’. (Patient 17, 32 years old, female) ‘The language is already ok. There's not much design and graphic. Not many human graphics inside’. (Patient 3, 35 years old, male) ‘For me, the colour needs to be more striking. The white background is not so attractive’. (Patient 5, 32 years old, male)
Theme 3: Reminder function is important for symptom monitoring	‘The reminder alert, it will be good if there's sound like WhatsApp’. (Patient 4, 35 years old, male) ‘Actually it (reminder) is good because sometimes I will always forget that I need to key in my symptoms at all’. (Patient 1, 23 years old, male) ‘Yes, the app did give the reminder, but sometimes I’ll just ignore it. Because when I get 100% (answered all questions without triggering alert to seek early treatment), I think I am ok already’. (Patient 11, 53 years old, female)

Usability

For usability, we discovered three themes:

Assistance with installation helped ensure application usage. Patient 14 informed that assistance in installing the application during recruitment helped to ensure he used the application; however, he would unlikely to install the application if he had to do it himself at home, mainly because he was unwell.

A more vibrant and attractive design was desired. The DengueAid application was designed with a white background and all the basic functions were displayed on the homepage (Figure 3). Our participants preferred a more vibrant and attractive design.

The reminder function is important for symptom monitoring. Our participants appreciated the reminder function in the application to help remind them to key in the symptoms. Patient 4 suggested including sound reminder as well. However, some participants would still ignore the reminder function.

Figure 3.

Screenshot of DengueAid application homepage.

Discussion

In our pilot study, we explored the feasibility of the research process (recruitment, data collection and follow-up), the preliminary clinical outcomes and the acceptability of the application in terms of utility and usability. We found some limitations in our recruitment, data collection and follow-up procedures. Improvement is needed for data collection to improve on the accuracy of the data and reduce recall bias. The DengueAid application was acceptable to our participants but further modification to the intervention is needed to improve the utility. The preliminary clinical outcome data suggested that the application in its current form was not effective in changing health-seeking behaviour.

A low recruitment rate is a common challenge encountered in RCTs. In this study, our recruitment rate of 37% was low in contrast to the average response rate of 70% reported in the literature for acute febrile illness.¹⁷ Compared to similar RCTs recruiting dengue patients being followed up as outpatients, Nitinai et al.¹⁹ and Nasir¹⁸ reported a much higher recruitment rate of 93.2% and 93.5%, respectively, after excluding those ineligible. Their refusal rates were between 3.9% and 6.8%, as compared to our refusal rate of 63%. It is worth noting that Nasir's study was conducted in Malaysia in a location adjacent to our study site, with a comparable population. From our limited data, some of the reasons cited for refusal included lack of interest, unfamiliarity with smartphone applications and family commitment. The low response rate is a challenge, but not unexpected. Patients with dengue fever generally felt unwell at the time of presentation and a request to use a new technology may be construed as an additional burden. This is different from Nitinai's and Nasir's studies where their intervention entailed oral drug and simple fluid chart respectively, which were arguably less cumbersome compared to digital application.^18,19 The physical condition of the patients may limit their interest to participate.²⁰ In retrospect, it may be worth collecting and analysing the characteristics of the patients who refused to participate. Use of digital applications is often lower in those with older age, lower education status, minority ethnic groups or limited Internet access.²¹ Our participants were noticeably of younger age with a mean age of 31 years old, predominantly of Malay ethnic, which is the major ethnic group in Malaysia apart from Chinese, Indian and other indigenous groups. The language barrier cited by an Indian patient as the reason for refusal during our recruitment is an example of how the digital application might have inadvertently excluded certain groups. Internet penetrability is high in our study population but understanding the adequacy of Internet bandwidth and mobile data limit is important. Such information would be useful to plan future recruitment strategies.

Improvising the recruitment strategies may help to improve the recruitment rate in our study. Our initial assumption was the adoption of digital applications could be done at home once the patients settled down and felt more comfortable, considering that the instructions could be delivered via phone. However, our pilot study demonstrated that face-to-face or direct recruitment yielded better success compared to phone recruitment. Difficulty to contact potential participants via phone was recognised as one of the challenges in recruitment for RCT.²² There may also be a trust or credibility issue when it comes to phone recruitment. Multi-pronged recruitment strategies utilising social media, text messages, websites, banners and flyers may help improve recruitment in RCTs but not all methods are suitable for acute infectious illnesses like dengue as the recruitment is time-sensitive.²³ Coupling links to recruitment websites with face-to-face and phone recruitment, along with the presence of banners and flyers at the study site may help increase the visibility and credibility of the study. Nevertheless, such measures would incur higher costs and need to be budgeted in.

For the data collection and follow-up in this trial, we designed the protocol to be flexible to accommodate the inherent variability of disease progression and management in dengue. For the data collection in our study, one of the major sources of data particularly for the outcome came from the verbal reporting from the patients upon questioning by RA at two weeks post-recruitment. As the follow-up interval differed among the patients and they may change the place of follow-up, we deemed contacting patients via phone after the follow-up concluded as the most feasible way. We were able to contact almost all our patients except one in the control arm. It is also important to avoid contact with patients during the active follow-up period as that may trigger them to use the application more frequently. A multi-source approach to collect data (medical records, application data and phone interviews) helped to improve data accuracy. Our protocol is not too different from previously reported RCTs for dengue managed as an outpatient. Nitinai et al.¹⁹ followed up their patients every other day for 14 days or until the treating doctors ended the follow-up, whichever was shorter, irrespective of the duration of illness upon recruitment. Their data collection was also done via telephone calls and data extraction from medical notes, similar to ours. We did not follow up with our patients every other day due to concern of contamination, where contact with patients may trigger them to use the application more frequently. Similarly, the variability in follow-up duration was also observed in a pilot RCT by Nasir et al.,¹⁸ which highlighted the challenge of standardising follow-up intervals for dengue patients. Although there was variation in the follow-up interval, the outcomes of interest in our study were still clinically meaningful, measurable and comparable at different time points. This would help to assess the impact of the intervention regardless of the follow-up interval. To mitigate the effect of the variability in follow-up, one consideration is to refine the inclusion criteria where only dengue patients presenting within the first three days of illness would be recruited. Dengue fever typically lasts for seven days with the critical phase often occurring after the third day of fever.⁵ Capturing the patients in the earlier phase would allow a more comparable starting point to reduce the variability, and increase chances of the application utility as warning symptoms usually appear during the critical phase. However, in our pilot study, the patients presented to the clinic only on Day 3–5 of illness. Tightening this inclusion criteria may reduce the recruitment rate further. Recruitment at multiple study sites would be needed to improve the chances of successful recruitment. Another area for improvement is perhaps the participants can be contacted as soon as the treating doctors end the follow-up, instead of waiting for two weeks to reduce recall bias, similar to what was done by Nitinai et al.¹⁹

We deemed the clinical outcome measures suitable, especially the main outcome of interest: the proportion of patients with warning symptoms who sought help early. However, there may be concern about overreporting of warning symptoms in DengueAid due to the availability of self-reporting via DengueAid. It may be questionable whether the proportion of patients with warning symptoms is comparable between participants in both arms. It is expected that verbal reporting of warning symptoms when questioned would be lower. This is shown in both arms of our pilot trial. Only four reported warning symptoms in the control arm while in the intervention arm, only nine out of 15 reported warning symptoms although data from the application showed that 14 of 15 in the intervention arm had warning symptoms. This may be due to recall bias or unawareness of warning symptoms. Poor knowledge of warning signs in dengue is not new as it has been shown that the general public could identify well the common symptoms of dengue fever such as high-grade fever, headache, arthralgia and myalgia but relate less to symptoms indicative of warning signs such as rapid breathing, restlessness and severe abdominal pain.²⁴ In this pilot trial, we reported the hospitalisation rate of those with warning signs in the intervention arm versus the control arm. A more useful outcome on hospitalisation rate would be the hospitalisation rate of those who came earlier to seek help versus those who did not, as an indicator of how specific the warning signs were.

A more extensive set of questions in the application as suggested by some of our participants, such as second-level questions to clarify the initial reported symptoms in the application, may reduce the number of warning symptoms triggered by the application. We have highlighted in the ‘Results’ section that non-specific symptoms such as difficulty in carrying out usual activities (the surrogate term for severe lethargy) and drowsiness were the most commonly reported symptoms in the application. In the literature, abdominal pain, persistent vomiting, mucosal bleeding and lethargy were reported as the common warning signs for severe dengue.^14,25 A multi-centre study on confirmed adult dengue deaths in Singapore showed that lethargy was the commonest warning symptom at presentation.²⁶ Therefore, it is important not to dismiss these non-specific symptoms. Severe lethargy is a warning symptom for dengue but general lethargy is expected in most patients with dengue fever. Thus, it is not surprising that most patients in our study would report that they had difficulties in carrying out their usual activities. The type of symptoms reported in the application, particularly the non-specific symptoms, may be a poor differentiator of the severity of dengue. A more detailed set of questions to follow-up on the initial reported warning symptoms may be useful to identify a ‘true’ warning symptom for dengue. The development and incorporation of a predictive algorithm comprising the combination of warning symptoms may help improve the prediction of severe dengue using this application.^14,25

We still uncovered utility and usability issues during the pilot testing phase despite the rigorous development process. While our participants had positive opinions of the application, the initial findings suggested suboptimal adoption of the symptom monitoring features and limited usefulness in changing health-seeking behaviour. In the intervention arm, less than half of the patients completed the symptom monitoring thrice daily and only one participant (out of 15) used the application daily until he completed his follow-up for dengue. Digital interventions often experience high attrition rates due to their reliance on self-directed and unstructured approaches.²⁷ The health condition might also affect the completion rate of the symptom reporting in our application. As shown in our qualitative interviews, our patients might skip the symptom reporting when they felt unwell, although those were the times that they might actually need further medical attention. The omission of symptom reporting in the application during the unwell period may also be due to the perceived lack of usefulness of the application by the patients. Further research may be useful to explore the optimal frequency of symptom monitoring, and ways to improve the perceived usefulness and adoption of digital health among patients with acute illnesses such as dengue. In our study, only three out of 14 patients with warning symptoms sought help early. The application, in its current form, does not appear to be effective in changing our participants’ health-seeking behaviour, although the sample size is too small for a definitive conclusion. A systematic review has shown that mobile-health technology usually has a limited impact on behavioural change.²⁸ Our development process of the application could have been more rigorous. Our beta testing was conducted on recovered dengue patients (due to safety and ethical concerns) instead of active, symptomatic dengue patients. Hence, we were unable to replicate the real-world circumstances during testing. As discussed earlier, the unwell condition of the symptomatic dengue patients is an important factor affecting the utility of the application. There are also many other confounding factors in our study such as the limited access to healthcare due to personal circumstances (such as the distance to the nearest healthcare facilities and availability of transport), and the occurrence of warning symptoms at night reducing the likelihood to seek help early. Previous literature had suggested that there were many factors interplayed in influencing the decision to seek help early in dengue fever and the decision did not depend on the presence of warning signs alone.^29,30

We also discovered some concerns in this pilot trial. One, the compliance to follow-up was lower in the intervention group. Patients may regard the application as a substitute for usual care although it was emphasised during recruitment that the application was not meant to replace usual care. Patients may get false assurance from the application and influence their decision for follow-up. Second, some participants had a false impression that the symptoms checker could be used as a diagnostic tool in the future to help them decide whether to seek initial treatment or not. The mechanism needs to be in place to disable the usage of the application after the patients complete their follow-up for dengue to prevent potential misuse.

The various issues we highlighted in this discussion, from the trial processes (such as recruitment, data collection and follow-up) to the limited utility and usability of the application, underscore the importance of involving patients together with researchers and clinicians in the co-design process of the research protocol and the intervention tool from an earlier stage. Patients typically participated in the later phases of research, such as usability testing, where their capacity to impact significant alterations were minimal.²⁸ Exploring patients’ priorities during recruitment and understanding that these priorities may shift with the provision of trial information by recruitment staff would help improve the recruitment rate and retention.³¹ Involving patients and the public in the design of the digital intervention has been shown to increase the user uptake and improve the effectiveness of the intervention.^32,33

Strength and limitations

To the best of our knowledge, our pilot trial was the first trial to evaluate the feasibility and preliminary effectiveness of a digital self-monitoring system for dengue patients. Our findings informed the need for modification to the existing self-monitoring system and provided important input to design a digital intervention RCT for acute infectious diseases with inherent variability such as dengue.

There are some limitations to our pilot study. First, we had only conducted the pilot trial in a single health centre and the clinical set-up of each clinic may differ and affect the recruitment method. Our findings may not be generalisable to the larger population. Second, the lack of data for reasons for refusal to participate limited our understanding which could have helped improve the recruitment process in future trials. Thirdly, some of our outcome data were obtained via phone interviews with the patients after they concluded the follow-up for dengue. There may be recall bias affecting the accuracy of the data. Fourthly, our questionnaire on symptom monitoring did not undergo the full validation process and the pilot testing of the questionnaire could have been more rigorous. As mentioned above, we only included recovered dengue patients in our testing where issues such as the need for a more extensive set of questions and limited utility to change the behaviour of patients were not identified during pilot testing.

Conclusion

This feasibility study highlights issues with low recruitment rate and difficulty in the data collection process due to the variable interval of care for acute dengue. There are many confounding factors affecting the utility and usability of the dengue self-monitoring application such as the unwell condition of the patient and access to healthcare. More extensive assessment of symptoms is needed in the disease monitoring algorithm. Co-design of the trial protocol and digital intervention involving important stakeholders such as the clinicians and patients is suggested to improve the recruitment strategies, data collection method, and modify the intervention to increase its uptake and effectiveness.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241277710 - Supplemental material for Feasibility and acceptability of a dengue self-monitoring system to reduce treatment delay in Malaysia: A single-centre pilot randomised controlled trial

Supplemental material, sj-docx-1-dhj-10.1177_20552076241277710 for Feasibility and acceptability of a dengue self-monitoring system to reduce treatment delay in Malaysia: A single-centre pilot randomised controlled trial by Wei Leik Ng, Chirk Jenn Ng, Chin Hai Teo, Tan Fong Ang, Yew Kong Lee, Haireen Abdul Hadi, De Min Chiang, Mohd Khairi Mohd Noor, Sharifah Faridah Syed Omar, Hang Cheng Ong, Pui Li Wong, Anjanna Kukreja, Thiam Kian Chiew, Sim Ying Ong and Abdul Muhaimin Noor Azhar in DIGITAL HEALTH

Supplemental Material

sj-doc-2-dhj-10.1177_20552076241277710 - Supplemental material for Feasibility and acceptability of a dengue self-monitoring system to reduce treatment delay in Malaysia: A single-centre pilot randomised controlled trial

Supplemental material, sj-doc-2-dhj-10.1177_20552076241277710 for Feasibility and acceptability of a dengue self-monitoring system to reduce treatment delay in Malaysia: A single-centre pilot randomised controlled trial by Wei Leik Ng, Chirk Jenn Ng, Chin Hai Teo, Tan Fong Ang, Yew Kong Lee, Haireen Abdul Hadi, De Min Chiang, Mohd Khairi Mohd Noor, Sharifah Faridah Syed Omar, Hang Cheng Ong, Pui Li Wong, Anjanna Kukreja, Thiam Kian Chiew, Sim Ying Ong and Abdul Muhaimin Noor Azhar in DIGITAL HEALTH

Footnotes

Acknowledgements

We thank the Director General of Health Malaysia for his permission to publish this article. We also appreciate the assistance of Dr Azah Abdul Samad, the consultant family medicine specialist from the Ministry of Health Malaysia in helping us with the execution of the pilot RCT.

Contributorship

CJN and CHT conceptualised the study. All authors contributed to the development of the research proposal, design of the mobile application and development of the trial protocol. WLN, CHT, DMC and MKMN executed the study, including material preparation, data collection and analysis. The first draft of the article was written by WLN and all authors commented on previous versions of the article. All authors read and approved the final article.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

This study was approved by the Universiti Malaya Medical Centre Medical Research Ethics Committee (MREC ID NO: 2022328-11112) on 13 June 2022, and the Medical Research and Ethics Committee, Ministry of Health Malaysia (NMRR ID-22-00813-UOM) on 26 August 2022. All participants provided written informed consent before enrolment in the study.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the University of Malaya Impact-Oriented Interdisciplinary Research Grant Programme, (grant number IIRG002B-2020HWB).

Guarantor

WLN

Trial registration

This pilot trial is registered with ClinicalTrials.gov (ClinicalTrials.gov Identifier: NCT05688748) with the trial name: Effectiveness Evaluation of a Dengue Self-monitoring System. The protocol can be accessed via the following link: .

ORCID iDs

Wei Leik Ng

Chirk Jenn Ng

De Min Chiang

Supplemental material

Supplemental material for this article is available online.

References

Stanaway

Shepard

Undurraga

, et al. The global burden of dengue: an analysis from the Global Burden of Disease Study 2013. Lancet Infect Dis 2016; 16: 712–723.

Messina

Brady

Golding

, et al. The current and future global distribution and population at risk of dengue. Nat Microbiol 2019; 4: 1508–1515.

Guo

Zhou

Wen

, et al. Global epidemiology of dengue outbreaks in 1990–2015: a systematic review and meta-analysis. Front Cell Infect Microbiol 2017; 7: 317.

Togami

Chiew

Lowbridge

, et al. Epidemiology of dengue reported in the World Health Organization's Western Pacific Region, 2013-2019. Western Pac Surveill Response J 2023; 14: 1–16.

Malaysia Ministry of Health. Clinical practice guideline of management of dengue infection in adults. 3rd ed., Putrajaya, Malaysia: Malaysia Health Technology Assessment Section, 2015.

World Health Organization. Regional Office for South-East Asia. Comprehensive guideline for prevention and control of dengue and dengue haemorrhagic fever. Revised and expanded edition. New Delhi, India: WHO Regional Office for South-East Asia; 2011 [1 September 2023], https://apps.who.int/iris/handle/10665/204894 .

Leo

Y-S

Gan

E-L

, et al. Utility of warning signs in guiding admission and predicting severe disease in adult dengue. BMC Infect Dis 2013; 13: 498.

Ang

Rohani

Look

. Role of primary care providers in dengue prevention and control in the community. Med J Malaysia 2010; 65: 58–62.

Moore

. The evolution of telemedicine. Future Gener Comput Syst 1999; 15: 245–254.

10.

Korsbek

Tønder

. Momentum: a smartphone application to support shared decision making for people using mental health services. Psychiatr Rehabil J 2016; 39: 167–172.

11.

Lim

Abdullah

, et al. Utility and usability of an automated COVID-19 symptom monitoring system (CoSMoS) in primary care during COVID-19 pandemic: a qualitative feasibility study. Int J Med Inform 2021; 155: 104567.

12.

Herbuela

Karita

Francisco

, et al. An integrated mHealth app for dengue reporting and mapping, health communication, and behavior modification: development and assessment of Mozzify. JMIR Form Res 2020; 4: e16424.

13.

Lwin

Vijaykumar

Fernando

, et al. A 21st century approach to tackling dengue: crowdsourced surveillance, predictive mapping and tailored communication. Acta Trop 2014; 130: 100–107.

14.

Ahmad

Ibrahim

Mohamed

, et al. The sensitivity, specificity and accuracy of warning signs in predicting severe dengue, the severe dengue prevalence and its associated factors. Int J Environ Res Public Health 2018; 15(9). doi:https://doi.org/10.3390/ijerph15092018

15.

Abrahamsson

Salo

Ronkainen

, et al. Agile software development methods: review and analysis. Proc Espoo 2002; 2002: 3–107.

16.

Julious

. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat 2005; 4: 287–291.

17.

Olson

Lamb

Lopez

, et al. Performance of a mobile phone app-based participatory syndromic surveillance system for acute febrile illness and acute gastroenteritis in rural Guatemala. J Med Internet Res 2017; 19: e368.

18.

Nasir

Mohamad

Lum

LCS

, et al. Effectiveness of a fluid chart in outpatient management of suspected dengue fever: a pilot study. PLoS One 2017; 12: e0183544.

19.

Nitinai

Nasomsong

Chuerboonchai

, et al. Effect of montelukast in preventing dengue with warning signs among patients with dengue: a multicenter, randomized, double-blind, placebo-controlled trial. PLoS Negl Trop Dis 2024; 18: e0011927.

20.

Gul

Ali

. Clinical trials: the challenge of recruitment and retention of participants. J Clin Nurs 2010; 19: 227–233.

21.

Roessler

, editor. The mobile phone revolution and digital inequality: scope, determinants and consequences, 2018.

22.

Baron

Hirani

Newman

. Challenges in patient recruitment, implementation, and fidelity in a mobile telehealth study. Telemed J E Health 2016; 22: 400–409.

23.

Pfammatter

Mitsos

Wang

, et al. Evaluating and improving recruitment and retention in an mHealth clinical trial: an example of iterating methods during a trial. Mhealth 2017; 3: 49.

24.

Selvarajoo

Liew

JWK

Tan

, et al. Knowledge, attitude and practice on dengue prevention and dengue seroprevalence in a dengue hotspot in Malaysia: a cross-sectional study. Sci Rep 2020; 10: 9534.

25.

Thein

Gan

Lye

, et al. Utilities and limitations of the World Health Organization 2009 warning signs for adult dengue severity. PLoS Negl Trop Dis 2013; 7: e2023.

26.

Leo

Thein

Fisher

, et al. Confirmed adult dengue deaths in Singapore: 5-year multi-center retrospective study. BMC Infect Dis 2011; 11: 123.

27.

Patel

Akhtar

Malins

, et al. The acceptability and usability of digital health interventions for adults with depression, anxiety, and somatoform disorders: qualitative systematic review and meta-synthesis. J Med Internet Res 2020; 22: e16228.

28.

Free

Phillips

Galli

, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013; 10: e1001362.

29.

Teo

Toh

, et al. Factors influencing healthcare seeking in patients with dengue: systematic review. Trop Med Int Health 2022; 27: 13–27.

30.

Toh

, et al. Self-care practices and health-seeking behaviours in patients with dengue fever: a qualitative study from patients’ and physicians’ perspectives. PLoS Negl Trop Dis 2023; 17: e0011302.

31.

Mills

Donovan

Wade

, et al. Exploring treatment preferences facilitated recruitment to randomized controlled trials. J Clin Epidemiol 2011; 64: 1127–1136.

32.

Birnbaum

Lewis

Rosen

, et al. Patient engagement and the design of digital health. Acad Emerg Med 2015; 22: 754–756.

33.

McCurdie

Taneva

Casselman

, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol 2012; Suppl: 49–56.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB

0.61 MB