Abstract
Objectives
To investigate the implementation of teleconsultation and assess the level of readiness for its adoption among various types of healthcare facilities
Methods
This cross-sectional study involved medical doctors working in a public hospital, a private hospital, and community health centers in Yogyakarta, Indonesia. We recruited 29 medical specialists from various departments in two hospitals and 27 heads of community health centers. The readiness items were categorized into sections that encompassed various readiness areas such as core, technological, motivational, learning, work culture, and policy readiness. Data were analyzed using a one-way analysis of variance and the Kruskal-Wallis test to evaluate differences in levels of readiness across healthcare facilities. A logistic regression analysis was conducted to further assess factors predicting the implementation of teleconsultation.
Results
Variations in technological readiness were observed between the community health centers and the public hospital (p = 0.006) and the private hospital (p = 0.007). Differences in learning readiness were found between the public hospitals and private hospitals (p = 0.01). There were also disparities in cultural readiness between the public hospital and the private hospital (p = 0.04) and between public hospital and community health centers (p = 0.01). Logistic regression revealed an association between technological readiness and the use of video teleconsultation (OR = 1.13; p = 0.017). The private hospital was more likely to implement video-based teleconsultation than was the public hospital (OR = 2.68; p = 0.003) or community health centers (OR = 3.13; p ≤ 0.001).
Conclusion
Significant differences in technology readiness were identified among community health centers, public hospitals, and private hospitals. Future policy implementation should focus on customizing technology use and providing cultural training to help healthcare institutions with different technological readiness levels.
Introduction
Indonesian healthcare facilities have made notable advancements in teleconsultation following the coronavirus disease 2019 (COVID-19) pandemic. They adjusted to changes in patient behaviors by integrating technology into their operations, including teleconsultations. Teleconsultations not only simplify patient consultations with medical professionals without the need for in-person visits but also expand services and increase revenues for healthcare facilities. 1 Furthermore, this shift towards telehealth services has improved healthcare accessibility, particularly in remote areas, ultimately enhancing overall healthcare delivery. 2
There are two forms of teleconsultation services that can be utilized by healthcare facilities. Communication between two teleconsultation participants that do not simultaneously occur, typically mediated by images and text, is referred to as asynchronous teleconsultation. On the other hand, synchronous teleconsultation involves audio and video and simultaneously occurs between two subjects. The choice between these modalities depends on factors such as the nature of the medical consultation and the availability of resources. 3
The Indonesian government actively contributed to promoting and implementing teleconsultation for healthcare services. Several regulations were issued that govern teleconsultations, including the “Minister of Health Regulation on The Provision of Telehealth for Healthcare Facilities.” This regulation encompasses governance, requirements, registration, financing, rights and responsibilities of teleconsultation practitioners, as well as oversight and supervision. 4 Another regulation is the “Minister of Health Decree on The Guidelines for Healthcare Services Through Telemedicine During the COVID-19 Pandemic.” This decree outlines technical aspects of COVID management, from self-health monitoring during isolation to treatment that can be facilitated through teleconsultation. 5
Implementation of teleconsultation measures should be preceded by an analysis of the readiness of systems. The readiness levels of both health personnel and health facilities should be assessed. Physicians, nurses, and dietitians who will potentially use teleconsultation should be assessed for their willingness, experience, financial background, and engagement with teleconsultation.6–8 Readiness analyses of healthcare institutions should cover areas such as core management, technology, engagement, motivation, learning, expectations, culture, and policies. Assessing these areas is crucial to obtain a comprehensive overview of a facility's readiness, from the managerial level to the resources the facility has and also the support it receives from the government. 9 Readiness analyses are expected to assist healthcare facilities in formulating better plans and strategies to implement teleconsultation. Readiness analyses also help healthcare professionals themselves be better prepared to effectively utilize teleconsultations. 10
Among various methods of assessment, in this research, we examined six different areas to evaluate readiness: core, infrastructure, motivation, learning, work culture, and policies. Core readiness represents the self-realization of health facility members that adopting teleconsultation can enhance their services and overcome limitations to conventional approaches. 11 Infrastructure, including hardware, software, and internet connections, plays vital roles in facilitating effective teleconsultations and ensuring safety. 12 Motivation is a crucial catalyst instigating shifts that prompt both healthcare personnel and the facility itself to promote change. 13 Learning readiness signifies the presence of experienced peers and training opportunities related to technological advancements. The work culture also plays a role in influencing the adoption of teleconsultation, for example, if a spirit of change permeates throughout the work environment. 9 These previously mentioned internal components are complemented by external factors, such as government policies. The presence of policies that are supportive, or even mandatory, for healthcare facilities, can accelerate the implementation and facilitate systemic adoption of teleconsultation. 14
The common model of healthcare institutions in Indonesia consists of primary health care (PHC), which consists of community health centers (CHCs), health clinics, and hospitals. These models differ in terms of organizational structures, management, and healthcare workforce compositions. CHCs aim to provide basic healthcare services and general practice, and manage community health, while hospitals focus on providing advanced healthcare services such as specialized practice, surgery, and intensive care. 15 Hospitals in Indonesia are classified into two types based on ownership: public and private hospitals. These two types differ in terms of management, with private hospitals focusing on generating profits to keep the hospital operational. 16 These factors could affect how healthcare institutions perceive the importance of teleconsultation (Table 1).
Distribution of healthcare facilities based on selected characteristics.
CHCs: community health centers.
Significant at p < 0.05.
Conducting research on telehealth readiness for healthcare facilities in Indonesia is imperative due to the evolution of healthcare delivery and the potential transformative impacts of telehealth technologies. Indonesia, with its diverse geographic and unique healthcare system, requires a nuanced understanding of healthcare facilities’ preparedness to effectively integrate telehealth into their services. There is limited research on this topic, and no study has yet compared differences among healthcare institutions. Addressing these issues is important for developing teleconsultation and identifying specific hurdles faced by healthcare facilities in Indonesia.
Given the above issues, in this study, we assessed the variability in the readiness among healthcare facilities in Indonesia to implement teleconsultation practices. By delving into distinct aspects of readiness, including core, technological, motivation, learning, work culture, and policy, we sought to provide an understanding of the factors influencing teleconsultation implementation through this research. Furthermore, we also aimed to establish associations between different aspects of readiness and the actual implementation of teleconsultation in healthcare facilities.
Methods
Study population
This study was a cross-sectional survey conducted in Yogyakarta from August 2022 to August 2023. The healthcare facilities selected for this study were healthcare facilities affiliated with the Faculty of Medicine Universitas Kristen Duta Wacana (FoM UKDW) and were in the same region as per the Indonesian National Health Insurance (INHI) regionalization. We purposively selected these facilities to ensure alignment with our research objectives, as they provide a diverse patient population and a comprehensive range of healthcare services. This selection allowed for a more representative analysis of healthcare practices and outcomes in the region. Subjects were medical specialists from various departments across two type B hospitals in Yogyakarta City and general practitioners (GPs) of CHCs in Bantul Regency. Type B hospitals are hospitals with at least 200 inpatient beds and which provide comprehensive basic specialization as well as several sub-specializations. 17 The inclusion criteria for this study were medical specialists of type B hospitals or a GP of a CHC, who had been working at their respective healthcare facilities for at least 1 year and were willing to participate in the research. The exclusion criterion was a receipt of an incomplete response. The departments or areas of practice of each respondent were not mentioned in this study to protect the identity of the respondents, as some departments only had one medical doctor. To prevent duplication, the questionnaire form could only be filled out once per email address.
Measures
The present study assessed the readiness of healthcare facilities to implement teleconsultation by adopting an e-health readiness assessment tool for healthcare institutions in developing countries developed by Khoja et al. (2007). This digital health readiness assessment tool comprises a targeted questionnaire and scoring system designed for managers and healthcare providers. Its successful application in Pakistan, 18 Botswana, 19 and Uganda 11 underscores its effectiveness in assessing e-health adoption readiness. In this study, we further customized the assessment tool to align with the diverse contexts of hospitals and CHCs in Indonesia, categorizing six distinct areas: core readiness (seven items), technological readiness (five items), motivational readiness (three items), learning readiness (three items), work culture readiness (five items), and policy readiness (four items). Authors have obtained permission from the original researcher, Dr Shariq Khoja who owns the questionnaire, to use and modify it according to the context of healthcare services in Indonesia.
The core readiness component encompassed understanding need assessments, comfort levels, and trust in technology, crucial planning aspects, and technology's compatibility with existing services. Technological readiness evaluated the availability, affordability, and access to required technology, hardware, and software, as well as capacity building. Motivational readiness assessed perceived benefits such as revenue enhancement and service quality improvement. Learning readiness focused on training programs, involving healthcare providers in planning and capacity building. Work culture readiness examined information sharing, collaborative patient management, staff and community technology adoption, and unified staff support. Policy readiness measured regulatory oversight, financial regulations, standardization, and local health department endorsements (Supplemental Table 1).
To ensure linguistic and conceptual equivalence of the questionnaire, we employed a back translation method. The original English version of the instrument was translated into Indonesian by a bilingual expert. Subsequently, a different bilingual expert, who was blinded to the original questionnaire, translated the Indonesian version back into English. Discrepancies between the original and back-translated versions were discussed and resolved by a panel of experts to ensure the final instrument accurately captured the intended constructs in the Indonesian context.
The questionnaire was then initially tested on 30 physicians who worked at FoM UKDW-affiliated health facilities. To ensure the validity of the data collection instruments, we conducted content validity and construct validity tests. Content validity was established through expert reviews by three senior researchers in the field of health information and management systems. Construct validity was assessed using convergent and discriminant validity assessment through confirmatory factor analysis (CFA). Convergent validity was assessed using the average variance extracted (AVE), where a value > 0.5 indicated strong convergent validity. The AVE serves as a common method in telehealth research for assessing convergent validity when questionnaires are employed as the research tool.20–22 Discriminant validity was demonstrated when the AVE value of a latent construct exceeded its squared correlation with any other latent construct, indicating that each construct shared more variance with its indicators than with other constructs in the model. Raykov's reliability coefficient (RRC) was utilized to ascertain the construct's reliability, requiring a coefficient value of at least 0.7 for a construct to be deemed reliable. The RRC, a relatively newer method compared to the alpha coefficient, has not been widely applied to telehealth readiness research. It is more suitable for testing multidimensional constructs, contrasting with the alpha coefficient, which is better suited for unidimensional constructs. 23 In summary, the AVE and RRC by latent variables ranged from 0.52 to 0.66 and 0.72 to 0.93, indicating acceptable construct validity and reliability. Results for these parameters are presented in Supplemental Table S1.
Statistical procedures
Descriptive statistics were summarized as numbers and percentages for categorical variables and means and standard deviations (SDs) for numerical values. Differences in each readiness area among healthcare facilities were assessed using a one-way analysis of variance (ANOVA) test for normally distributed variables and the Kruskal-Wallis test for those with a non-normal distribution. Post-hoc analysis with the Bonferroni correction was applied to ANOVA results to pinpoint specific group differences. A logistic regression analysis was conducted to further assess factors predicting the implementation of teleconsultation. All hypotheses were tested with a significance level set to p < 0.05, and odds ratios (ORs) and 95% confidence intervals (CIs) were given. All analyses were conducted using STATA version 17. 24
Ethical considerations
This research passed ethical clearance by the Research Ethics Committee of Universitas Kristen Duta Wacana (Yogyakarta, Indonesia) with ethical clearance letter no. 1435/C.16/FK/2022. Informed consent was obtained from all individual participants included in the study. Participants were provided with a detailed explanation of the study on the first page of the online questionnaire, which included information about the study's objectives, procedures, potential risks, and benefits. After reviewing this information, participants indicated their willingness to participate by answering mandatory consent questions. Participation in the study was entirely voluntary, and participants were informed that they could withdraw from the study at any time without any repercussions. Respondent identities were kept confidential by avoiding mention of their names and the departments where they worked.
Results
Prior to the commencement of the study, our aim was to assess the readiness of teleconsultation in hospitals and districts based on the preparedness of each department and CHC, with each hospital comprising 20 departments and 27 CHCs within the Bantul Regency. For the private hospital, 15 departments have filled out the questionnaire (75% response rate), while for the public hospital, only 14 departments filled out the questionnaire (70% response rate). As for CHCs, all of CHCs have filled out the questionnaire (100% response rate). One response from the CHC was incomplete, but the same CHC resubmitted the corrected questionnaire.
Most healthcare workers were in the age range of 41–50 years (62.5%). We found that there were differences in the adoption of teleconsultation based on the type of healthcare facility (p < 0.05). There was no association between the average age range and the adoption of teleconsultation by healthcare facilities.
The distribution of healthcare facilities’ readiness is shown in Table 2. Neither core nor policy readiness was normally distributed, hence we used the Kruskal-Wallis test instead of a one-way ANOVA. There were significant mean differences in technological, learning, and work culture readiness aspects (p < 0.05). Further assessment using pair-wise comparisons (Table 3) showed that both the public (mean difference −3.43, p < 0.05) and private hospitals (mean difference −3.39, p < 0.05) demonstrated better results than CHCs in terms of technological readiness. In terms of learning readiness, the public hospital was better prepared than CHCs (mean difference −2.01; p < 0.05). The public hospital also showed better work culture readiness compared to the private hospital (mean difference 2.69; p < 0.05) and CHCs (mean difference −2.73; p < 0.05).
Distribution of healthcare facilities’ readiness.
SD: standard deviation. a One-way ANOVA test. b Kruskal-Wallis test.
Significant at p < 0.05.
Pair-wise comparisons of technological, learning, and work culture readiness among different healthcare facilities.
Significant at p < 0.05; post-hoc test with the Bonferroni correction.
Table 4 presents an exploration of the association between each readiness domain and the implementation of teleconsultation. From six domains, technological readiness was shown to be associated with video-based teleconsultation (OR = 1.13; 95% CI = 0.20–2.06; p = 0.017).
Logistic regression of factors associated with text-based and video-based teleconsultation adoption.
Significant at p < 0.05.
Discussion
The utilization of telehealth by healthcare facilities was indirectly driven by the COVID-19 pandemic. Most CHCs implemented asynchronous teleconsultation, although only a few used video teleconsultations in their daily healthcare services. Text-based teleconsultation at CHCs was facilitated by the Social Security Agency for Health (Badan Penyelenggara Jaminan Sosial Kesehatan, BPJS Kesehatan) since 2021 through the teleconsultation feature in the pCare application. This application is mandatory for CHCs when providing services to patients with BPJS Kesehatan insurance. 5 This utilization can help improve the number of patient visits to CHCs and assist them in BPJS credentialing assessments. In contrast, teleconsultation development in hospitals is left to each hospital's discretion. The reasons behind some CHCs not using text-based teleconsultation in this study need further investigation, whether it is due to the CHC staff's unfamiliarity with the technology or if the local community is unaware of the service's availability.
This study found a significant difference in the technology readiness area between both types of hospitals and CHCs. Several components of technology readiness exhibited differences, including computer standards, internet access, and technology-related training, available in hospitals. Since 2013, the Ministry of Health has mandated that hospitals have their own hospital management information system (Sistem Informasi Manajemen Rumah Sakit, SIMRS). The complexity of SIMRS has led to higher computer and internet connectivity standards for healthcare professionals in hospitals. 25 This is not the case in CHCs, which are currently only required to use the web-based application pCare from BPJS Kesehatan. This application is relatively simple to use, so the required computer and internet connectivity standards are not as high, given the widespread distribution of CHCs in remote areas of Indonesia.
Learning readiness in the implementation of teleconsultation consists of a healthcare facility's experience in conducting training related to new technology and the staff's expertise in teleconsultation. Hospitals usually have more experience than CHCs in providing training related to technology usage. This is associated with numerous technological innovations, both in management and new services, that primarily focus on hospitals instead of CHCs. 26 Training and guidance for CHC staff often rely on the local health department or BPJS Kesehatan, although it is possible for CHCs to conduct their own training sessions. 27
This research focused on type B hospitals in Yogyakarta City, encompassing both government and private hospitals. Nevertheless, significant differences in work culture readiness were observed between the public hospital and private hospital, potentially stemming from variations in their vision, mission, and values. The hospital's developmental focus potentially influences the curiosity of its staff to delve into teleconsultation. 28 Another internal factor in healthcare facilities that can influence cultural readiness is resistance from healthcare staff who refuse to engage in teleconsultation. This resistance can arise from staff's lack of awareness about teleconsultation or reluctance to disturb the status quo. 29
The association between technology and the utilization of teleconsultation can be explained by the need for more complex equipment in video-based teleconsultation compared to text-based teleconsultation. Video teleconsultation requires additional hardware such as cameras and headsets, as well as rapid and stable internet connections. Existing regulations in Indonesia currently do not specify minimum requirements for conducting legal teleconsultation. A recent “Health Minister Decree on Telemedicine” only mentions that teleconsultation systems must be registered with the Ministry of Health, support interoperability with existing systems, and be able to protect data security. 4
Limitations
This study only sampled two type B hospitals, and there were some departments in each hospital that refused to fill out the research questionnaire. Further research with a broader sample from other hospitals should be conducted to compare readiness differences between different types of hospitals and various departments within hospitals. As this questionnaire is a modified version tailored for the Indonesian healthcare setting, future research should consider reassessing the internal validity considering the updated context of healthcare services in Indonesia. The research questionnaire used in this study had an AVE value slightly above the ideal threshold. The researcher retained some questions despite having low item factor loadings because they were deemed to differ among the healthcare facilities. This suggests lower discriminant validity, even though its face and convergent validity are excellent.
Conclusions
Significant differences were found between CHCs and both a public and a private hospital in terms of technology readiness. Additionally, differences were also observed between the public hospital and private hospital, as well as between the public hospital and CHCs, in terms of cultural readiness, and between the public hospital and CHCs in terms of learning readiness. The study also revealed an association between technological readiness and video teleconsultation. Future policy implementation should focus on customizing technology use and providing cultural training to help healthcare institutions improve different technological readiness levels. The adaptation of the teleconsultation readiness assessment instrument to Indonesian settings still needs to be further assessed. Further studies on healthcare workers’ acceptance are needed to thoroughly evaluate the potential of telehealth implementation in Indonesia.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241278296 - Supplemental material for Exploring disparities of teleconsultation readiness: A comparative analysis of healthcare facilities in Indonesia
Supplemental material, sj-docx-1-dhj-10.1177_20552076241278296 for Exploring disparities of teleconsultation readiness: A comparative analysis of healthcare facilities in Indonesia by Daniel Chriswinanto Adityo Nugroho, Katherina Adisaputro and Mitra Andini Sigilipoe, Ida Ayu Triastuti, Suryani Hutomo, Ardhinta Bellano Septarda, Aryososepti Yuwono, Caraka Randi Yusuf, Dedik Sulistiawan, Balqis Gusbela, Jason C Hsu, Emily Chia-Yu Su in DIGITAL HEALTH
Supplemental Material
sj-docx-2-dhj-10.1177_20552076241278296 - Supplemental material for Exploring disparities of teleconsultation readiness: A comparative analysis of healthcare facilities in Indonesia
Supplemental material, sj-docx-2-dhj-10.1177_20552076241278296 for Exploring disparities of teleconsultation readiness: A comparative analysis of healthcare facilities in Indonesia by Daniel Chriswinanto Adityo Nugroho, Katherina Adisaputro and Mitra Andini Sigilipoe, Ida Ayu Triastuti, Suryani Hutomo, Ardhinta Bellano Septarda, Aryososepti Yuwono, Caraka Randi Yusuf, Dedik Sulistiawan, Balqis Gusbela, Jason C Hsu, Emily Chia-Yu Su in DIGITAL HEALTH
Footnotes
Acknowledgements
The authors would like to express our sincere gratitude to the board of directors from both hospitals and the Bantul District Health Office for the permission and opportunity granted to conduct this research in their respective health facilities.
Contributorship
DCAN developed the research concept, compiled, and finalized the manuscript, integrating insights from all contributing authors. KA, MAS, and ABS were involved in data collection at both hospitals, while IAT, SH, AY, and CRY participated in data gathering across CHCs within the Bantul district. Expertise in statistical analyses and adherence to Indonesian health regulations were generously provided by DS and BG. JCSH and ECYS assisted in the drafting phase and supervised the overall research. The article has received a thorough review and unanimous approval for publication from all involved authors.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
The research passed ethical clearance by the Research Ethics Committee of Universitas Kristen Duta Wacana Yogyakarta, Indonesia (1435/C.16/FK/2022).
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Science and Technology Council in Taiwan, Ministry of Science and Technology in Taiwan, and Universitas Kristen Duta Wacana (grant numbers NSTC113-2221-E-A49-193-MY3, MOST111-2628-E-038-001-MY2, 283/J.05/FK/2021).
Guarantor
ECYS and JCSH.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
