The usability testing of the integrated electronic healthcare services for diabetes mellitus patients during the pandemic in Indonesia

Abstract

Introduction

Diabetes mellitus is a degenerative disease that obliges patients to take continuous healthcare services. During a pandemic situation, all access to healthcare facilities becomes limited. A web-based integrated electronic-healthcare system (IeHS) study was used to overcome this problem. This study aimed to describe patients’ ability to access and understand this web-based application.

Method

An observational study using a web-based IeHS took place in Muara Tebo, Indonesia, in 2021. A total of 82 outpatients with diabetes mellitus participated in this study. These patients belonged to local community-based diabetes mellitus. Only adult patients accustomed to using smartphones were invited to participate in this study. All participants were taught to access the web-based application through video recordings. They were asked to fill out the form regarding their understanding of the web-based integrated e-healthcare application. This form was sent online.

Results

From the calculation of the usability scale, the results show that patients have not been able to take advantage of this application in life (the system usability scale (SUS) score 63.38). There is no significant difference between patients’ characteristics and the SUS score.

Conclusion

Participants’ condition in this study reflects the general state of outpatients. The ability of these participants to access an internet-based application was related to their education level. This condition becomes a challenge for policymakers to improve local human resources to increase their health literacy.

Keywords

Diabetes mellitus My-Iehs community-based the system usability scale score web-based application

Introduction

The fallout from the pandemic coronavirus disease 2019 (COVID-19) is not just limited to the infected individuals and significantly influences patients with chronic diseases. In the current crisis, however, there is a fear of exposure in public environments because of the possibility of respiratory droplet-borne rapid virus spread. Over 463 million people have diabetes worldwide and 163 million in the Western Pacific Region; by 2045, this number will increase to 212 million. Indonesia is one of the 39 countries and territories of the Western Pacific International Diabetes Federation region. The cumulative cases of diabetes in adults in Indonesia is more than 10 million people (6.2%).^1,2

The success of a diabetes mellitus (DM) treatment is very much influenced by the individual, such as knowledge of the disease and adherence to treatment. Integrated healthcare services is a way to increase patients’ adherence and reduce the adverse effect of medication. The National Assessment of Adults Literacy in the United States defines health literacy as someone who can search, find, understand, and assess health information from electronic sources and apply the knowledge obtained to overcome or solve health problems.³

With the complexity and prevalence of non-adherence in the treatment of DM, integrated care with the use of cellular technology is increasingly appreciated and is a trend in recent years. Technologies with a special focus on patient needs and ease of use in access play an important role in the development of e-Health or electronic health and m-health or mobile health.^4–6 In Indonesia, the level of public awareness about health continues to increase. Based on data from the Indonesian Internet Service Providers Association for the Indonesian people, as much as 51.06% of users conduct internet searches in the health sector, and 14.05% consult with health experts.⁷ The users of health applications and sites are generally dominated by young people who live in cities, have a good level of education, and use the internet to find their own health solutions.⁸

A web-based application named My-Iehs has already been introduced to increase medication adherence in chronic disease patients. My-Iehs or Integrated e-Health Service is a web-based health service system application, with health services directly provided by the healthcare provider. My-Iehs was created with the idea to combine electronic health records containing laboratory values and demographic data.⁹ This study aimed to find out whether My-Iehs is easy enough for DM patients who will use the product. Furthermore, we provide an overview of the characteristics of respondents that can affect the level of understanding of the use of internet applications and introduce web-based health applications to the public so that they are expected to provide more effective and efficient medical services for both patients and healthcare providers.

Method

Research design and setting

DM has become one of the biggest contributors to the disease burden, especially among adults in Indonesia. Diabetes treatment in primary care facilities at community health centers (puskesmas) is still substandard. The poor outcome of this treatment is caused by health workers’ limited knowledge of health services about diabetes. It is exacerbated by the low level of public knowledge about the use of drugs.¹⁰

This research was a cross-sectional observational study in Muara Tebo, Indonesia, in 2021. This research followed the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guideline in conducting the study.^11,12

Participants, inclusion criteria, and instruments

The research was conducted through a questionnaire survey. This questionnaire was distributed directly to the chronic disease management program community on social media in Jambi province. This research was performed using a survey method through the distribution of questionnaires. This study was conducted to measure the level of usefulness of the My-Iehs application on participants (type 2 DM outpatients). The usability testing of this application is carried out using the system usability scale (SUS) questionnaire that has been validated (Cronbach's alpha = 0.841).¹³ The participants fill out the integrated informed consent via an online Google form. Only participants who filled out the informed consent can take part in this study. Participants followed the steps on the online Google form before completing the questionnaire. There is a link to access instructions in the form of videos on the Google form. Participants were asked to do all the instructions on the video. This video recording is 2 min long. The Indonesian language was used to explain instruction. The questionnaire was distributed during the year 2021. Only participants who followed the instructions correctly according to the video sent were classified on the inclusion criteria. The most decisive step in determining included participants is the duration of watching the video recording. Only participants who watched the video to the end were included in the inclusion criteria. This research is integrated into the usability study protocol of IeHS in Indonesia with registration number NCT04179344, which can be assessed at https://clinicaltrials.gov/ct2/show/NCT04179344.¹⁴

The sending and distribution of the questionnaires were carried out online using the Google form application; where there were four parts of the questionnaire, namely the first part containing the researcher's self-introduction, the introduction of the My-Iehs application, and the patient's consent to become a participant, the second part containing the identity and characteristics of the participants, the third part containing the step-by-step video. The My-Iehs application's steps are on the patient menu, the My-Iehs application link, and a username and password to enter the application. Participants should finish the task on the My-Iehs application until the complaint (on the chat box) is sent, as seen in Figure 1. The fourth part contains the SUS questionnaire about the usability test of the My-Iehs application.

Figure 1.

The task flow of this study for diabetes mellitus (DM) participants.

Sample size

Germas data from Jambi Province shows around 965 patients with DM in Muara Tebo Regency.¹⁵ Meanwhile, prolanis membership is around 10%.¹⁶ Purposive sampling was used in this research. Questionnaires were sent to prolanis members, as many as 100 people. Those who returned a response were recorded on the Google form as many as 82 people (response rate 82%). This number is no longer increased (saturated) at the end of 2021.

The SUS equation and interpretation

This research uses a web-based application that can be accessed via http://my-iehs.com/. All entries in the questionnaire will be stored directly in the researcher's Google Drive. Data were processed using the SUS score. The SUS is a questionnaire that can be used to measure the usability of a computer system according to the user's subjective point of view. The SUS can be used easily because the result is a score of 0 to 100 and does not require complex calculations. The equation of the SUS score follows: first, add up the overall score for all odd-numbered queries, then subtract five from the queries to get (X). Secondly, add the total score for other questions and then subtract that total from 25 to get (Y). Next, add up the total score of (X + Y) and multiply it by 2.5. This value would be the SUS score for an individual participant.¹⁷ A total score of >68 is considered to be above average. The SUS scores were evaluated and interpreted with the classes of grading scales. A score <51 goes into a very bad rating, a score of 51 to 68 goes into a bad rating, a score of 68 goes into a pretty good rating, a score of 68 to 80.3 goes into a good rating, and a score that is more than 80.3 goes into an extraordinarily good rating.¹⁷ Furthermore, SUS is free of charge, does not require additional fees, and is proven valid and reliable even with a small sample size.¹⁷

Variables and statistical analysis

This study had several independent variables (sociodemographics, disease, and family history of DM) or dependent variables (the SUS score). The author makes the background sociodemographic variable a predictor. The predictor that affects the significance value of the SUS score is a confounding factor. Since there is only the SUS score as the outcome variable, the Spearman correlation was used to analyze the correlation between independent and dependent variables.¹⁸

Bias

In this study, only smartphone patients can participate in the questionnaire survey, so they do not represent all DM prolanis members. This situation could lead to selection bias. There were several points on how the researcher saw the participants’ trustworthiness to overcome biases. First, it can be seen from the starting point of filling out the questionnaire after filling out the informed consent. The video duration is 2 min. Second, if the participant fills out the questionnaire before the first 2 min, it can be ascertained that the participant is not honest. Trustworthiness is guaranteed by the time record, which is also integrated into the Google form. This process can avoid any recall bias in respondents.

Results

The usability test on the My-Iehs application was carried out on 82 respondents who were 30 to 60 years old, as seen in Table 1. First, in addition, the usability test of this application was carried out on type 2 DM outpatients in Muara Tebo with the appropriate criteria. Only participants who followed the instructions correctly according to the video sent were classified on the inclusion criteria.

Table 1.

Participants’ characteristics.

No	Characteristic	N (%)	Mean (±SD)
1.	Age (years)		50.09 (±6.95)
	30–40	9 (11.00)
	41–50	31 (37.8)
	>51	42 (51.20)
2.	Gender
	Male	45 (54.90)
	Female	37 (45.10)
3.	Education
	Elementary	3 (3.70)
	Middle school	15 (18.30)
	High school	15 (18.30)
	University	49 (59.80)
4.	Occupation
	Civil servant	28 (34.10)
	Private sector	30 (24.40)
	Other	34 (41.50)
5.	DM duration
	<6 months	7 (8.50)
	6–12 months	18 (22.00)
	12–24 months	29 (35.40)
	>24 months	28 (34.10)
6.	DM medicine consumption
	<6 months	7 (8.50)
	6–12 months	18 (22.00)
	12–24 months	29 (35.40)
	>24 months	28 (34.10)
7.	DM dosage form
	Injection	26 (31.70)
	OAD	40 (48.80)
	Combination Injection and OAD	16 (19.50)

DM: diabetes mellitus; OAD: oral antidiabetic drug.

From the results in Table 2, it was found that there was no relationship between age, gender, education, occupation, and duration of using diabetes medication. Since there were no significant results in this study, confounding factors were difficult to obtain. Nonetheless, the previous research found a relationship between education and understanding of internet search applications because participants can learn how to use the internet at school, and formal education affects participants’ critical thinking skills.¹⁷

Table 2.

The relationship between participant characteristics and the SUS score.

No	Characteristic	N	SUS score (mean ± SD)	p-value
1.	Age (years)
	30–40	9	65.00 (±9.10)	0.74
	41–50	31	64.19 (±10.09)
	>51	42	62.44 (±13.11)
2.	Gender
	Male	45	63.78 (±12.92)	0.74
	Female	37	62.91 (±9.83)
3.	Education
	Elementary	3	61.67(±8.87)	0.60
	Middle school	15	66.50 (±13.05)
	High school	15	60.83 (±11.36)
	University	16	63.32 (±11.40)
4.	Occupation
	Civil servant	3	63.04 (±12.92)	0.357
	Private sector	15	66.50 (±11.37)
	Other	15	61.84 (±11.05)
5.	DM duration
	<6 months	7	58.93 (±9.88)	0.670
	6–12 months	18	62.50 (±11.63)
	12–24 months	29	63.62 (±12.45)
	>24 months	28	64.82 (±11.22)
6.	DM medicine consumption
	<6 months	7	58.93 (±9.88)	0.670
	6–12 months	18	62.50 (±11.63)
	12–24 months	29	63.62 (±12.45)
	>24 months	28	64.82 (±11.22)
7.	DM dosage form
	Injection	26	64.13 (±12.60)	0.712
	OAD	40	63.75 (±11.63)
	Combination Injection and OAD	16	61.25 (±10.00)
	Overall	82	63.38 (±11.57)

SUS: system usability scale; DM: diabetes mellitus; OAD: oral antidiabetic drug.

After analyzing the data, calculations are carried out to obtain conclusions from the research that has been done. Analyses were carried out using the SUS method, which started by adding up all the scores for each question from each participant. After obtaining the total score from each participant, the next step is to divide the total score by the number of existing participants to get an average score for all participants.

Based on data processing as tabulated in Table 2, an average SUS score of 63.38 was obtained, meaning it has a value scale of “D”; thus, the My-Iehs application has poor usability or rating, according to the participants in this study. Even after being broken down into several categories, the participants’ SUS scores in this study were still in the 60s. Therefore, this calculation still shows a bad value. Although the SUS score in this study was not very good, the correlation between the characteristics of the participants and the sus score itself was not significant (p-value > 0.05).

This result is related to the level of health literacy in chronic diseases in Indonesia, which is still low.^19,20 On the other hand, health literacy is very important for people with DM. Increasing health literacy is considered one of the most important actions influencing participants’ ability or interest in dealing with health problems.^3,19,21,22

Strength and limitation

This study has several advantages in the field of health security. All services are integrated, so vertical and siloed approaches are avoided to promote sustainability, efficiency, and effectiveness at the national and subnational levels. Strong, comprehensive health systems enable health security while, in turn, health security facilitates health system strengthening. This integrated and multisectoral systems-strengthening approach yields resilient health systems capable of providing quality universal health coverage, more equitable health outcomes, and improved well-being across populations. In addition to the personal ability to access the digital world, there are some drawbacks to using digital technology, such as the need for a stable internet network, reduced social interaction, adequate technology equipment, and being slow means being left behind.

Conclusion

There is no significant relationship between education, occupation, and family history of diabetes in understanding internet applications p-value >0.05. Based on the research data analysis, the My-Iehs application's usability level obtained a SUS score of 63.38 and was then interpreted as a D rating, which means bad. So it is necessary to improve the My-Iehs application to further maximize its function of the My-Iehs application as therapy support.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231173227 - Supplemental material for The usability testing of the integrated electronic healthcare services for diabetes mellitus patients during the pandemic in Indonesia

Supplemental material, sj-docx-1-dhj-10.1177_20552076231173227 for The usability testing of the integrated electronic healthcare services for diabetes mellitus patients during the pandemic in Indonesia by Najmiatul Fitria, Lusiana Idrus, Ayuthya Rizky Putri and Yelly Oktavia Sari in DIGITAL HEALTH

Footnotes

Authors' Contributions

NF is the principal investigator in this study. NF designs ideas and techniques for doing research. LR writes the protocol, and ARP and YOS collect and rewrite medical record data on worksheets. Furthermore, NF also performs data analysis and writes the manuscript.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: NF and YOS are lecturers from Universitas Andalas, Padang, Indonesia.

Ethical approval

Ethical approval was acquired from The Research Ethics Committee of Medical Faculty, Andalas University (No. 646/UN.16.2/KEP-FK/2021).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported and sponsored by the program of the Basic Research Grant Universitas Andalas with the contract number T/68/UN.16.17/PT.01.03-IS-RD/2021.

Guarantor

The faculty of Pharmacy Universitas Andalas is the guarantor of this research.

ORCID iD

Najmiatul Fitria

Supplemental material

Supplemental material for this article is available online.

References

International Diabetes Federation. IDF Western Pacific members [Internet]. 2020 International Diabetes Federation. 2020. Available from: https://idf.org/our-network/regions-members/western-pacific/members/104-indonesia.html.

Fitria1

Sari

Putry

, et al. Future challenge on probiotics uses from fermented milk on the endocrine disorder in human. IOP Conf Ser Earth Environ Sci 2021; 888: 012047.

Prose

But

Among

. National Assessment of Adult Literacy (NAAL) a first look at the literacy of America’s adults in the 21st century average prose and document literacy scores rise for Blacks and Asians but decrease Hispanics. 2006.

Jendle

. The use of eHealth for the care of patients with diabetes in connection to the COVID-19 pandemic. J Diabetes Sci Technol 2020; 14: 739–740.

Banerjee

Chakraborty

Pal

. Teleconsultation and diabetes care amid COVID-19 pandemic in India: scopes and challenges. J Diabetes Sci Technol 2020; 14: 714–715.

Toft

. Learning from COVID-19 and looking to technology ahead. J Diabetes Sci Technol 2020; 14: 801–802.

Pelaksanaan Praktik Kedokteran. Policy brief. Kebijakan dan Regulasi tentang Pelayanan Kesehatan Berbasis Sistem Elektronik atau Digital di Indonesia. 2017.

Duplaga

. A cross-sectional study assessing determinants of the attitude to the introduction of eHealth services among patients suffering from chronic conditions. BMC Med Inform Decis Mak 2015; 15: 33.

Idrus

Fitria

Hasan

, et al. PIN53 the usability testing of the integrated E-healthcare services (IEHS) web-based application in the therapy management of HIV and tuberculosis in Indonesia. Value Heal Reg Issues [Internet] 2020; 22: S57.

10.

Seuring

Rhode

Rogge

, et al. Using peer education to improve diabetes management and outcomes in a low-income setting: a randomized controlled trial. Trials 2019; 20: 548.

11.

Cuschieri

. The STROBE guidelines. Saudi J Anaesth 2019; 13: 31–34.

12.

von Elm

Altman

Egger

, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg 2014; 12: 1495–1499.

13.

Sharfina

Santoso

. An Indonesian adaptation of the system usability scale (SUS). In: 2016 international conference on advanced computer science and information systems, ICACSIS, 15-16 October 2016, Malang, 2017. pp. 145–148.

14.

Idrus

. No Title [Internet]. Usability study of IeHS in Indonesia. 2019 [cited 2022 May 19], https://clinicaltrials.gov/ct2/show/NCT04179344.

15.

Healthy living community movement (Germas). Health profile 2019. Jambi: Jambi Governmental, 2019.

16.

BPJS Kesehatan. Info BPJS. Implementasi prolanis di masa pandemi Covid-19. Jakarta: BPJS Kesehatan, 2020.

17.

Lewis

Sauro

. Item benchmarks for the system usability scale. J Usability Stud 2018; 13: 158–167.

18.

Andy

. Discover statistics using SPSS-third edition. Rev Mex Biodivers. 2011; 82: 179–191.

19.

Schillinger

Grumbach

Piette

, et al. Association of health literacy with diabetes outcomes. J Am Med Assoc 2002; 288: 475–482.

20.

Soemitro

. Analisis tingkat health literacy dan pengetahuan pasien hipertensi di puskesmas kabupaten Malang. J Ilm Mhs Univ Surabaya 2014; 3: 1–13.

21.

Williams

Baker

Parker

, et al. Relationship of functional health literacy to patients’ knowledge of their chronic disease. Arch Intern Med 1998; 158: 166–172.

22.

Novianti

Riyanto

. tingkat literasi media remaja desa dalam pemanfaatan internet. Tingkat Literasi Media Remaja Desa dalam Pemanfaat Internet 2018; 16: 159–171.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB