Delphi Consensus Study on Technical Skills for Doctor-to-Doctor Telemedicine Conferences and Consultations in Asia

Abstract

Introduction:

Health care disparities are severe across Asia. To bridge gaps in professional knowledge and skills, Doctor-to-Doctor (DtoD) telemedicine shows promise. However, the specific technical skills required for effective DtoD conferences and consultations, particularly those involving advanced technologies, remain undefined. This study aimed to establish a consensus on these essential skills.

Methods:

We used the Delphi method, engaging 30 participants from Asia-Pacific Advanced Network Medical Working Group DtoD training programs. Investigators and an expert panel developed an initial list of technical skills for both synchronous and asynchronous DtoD teleconferences and consultations, which participants evaluated in Round 1. Following an in-depth workshop, a revised questionnaire incorporating new skills and feedback was assessed in Round 2, yielding the final list.

Results:

The study included 30 hospital IT staff, doctors, and nurses from 11 Asian countries, 57% of whom were affiliated with university hospitals. All participants completed both rounds. After integrating 46 written comments, 10 new skills were added, resulting in a final list of 43 technical skills. These encompassed conceptual understanding, technical execution, coordination, and security policy. Notably, “Extracting images from medical devices” and “Maintaining audio quality” were identified as key skills.

Discussion:

This study is the first to delineate the technical skills necessary for DtoD telemedicine conferences and consultations in Asia. The identified skills can guide future training, assessment, and development initiatives to strengthen the telemedicine workforce across the region.

Keywords

Asia Delphi panel study doctor-to-doctor telemedicine medical education skill development training program

Introduction

While medical innovations contribute to advancing health services, noticeable disparities exist in health and medical services worldwide.¹ In Asia, disparities in health care services have been consistently reported.^2–5 One significant factor contributing to these disparities is lack of a skilled health workforce.^6–9 Traditionally, to learn the latest techniques, skills, and approaches to technology use, medical professionals visit overseas medical facilities. However, considerable time and costs are associated with physical travel. To overcome this limitation, doctor-to-doctor (DtoD) telemedicine conferences and consultations using information and communication technology are used to provide training at a distance. Evidence suggests that DtoD telemedicine can effectively bridge current gaps in health care by equipping health care professionals with diagnostic and therapeutic knowledge and skills.^10–13 The Coronavirus disease-2019 (COVID-19) pandemic further engendered the use of teleconferencing systems, with DtoD telemedicine conferences and consultations having been held on diverse topics, including COVID-19 control measures.^14–18 DtoD telemedicine includes remote imaging support such as teleradiology and telepathology, supporting remote consultation, tele-ICU, information sharing, and educational activities.¹⁹ These activities are based on video/audio communication and image output from medical devices, while some involve direct connection between medical devices such as microscopes in telepathology and robotic surgical equipment in teleoperation.

Routine adoption of DtoD telemedicine began in the 1990s, overcoming earlier cost and technical barriers since its inception in the 1960s.²⁰ In Asia, early initiatives included activities using Integrated Services Digital Network (ISDN) and satellite communications to link medical centers within Asia and globally.^21–24 Around 2000, the adoption of internet-based platforms expanded these activities to include case conferences and live demonstrations.^25,26 Live demonstrations, an educational method delivering procedures in real time, have become a popular instructional method in fields such as gastrointestinal surgery, cardiac surgery, endoscopy, and dentistry.^27–30 In Asia, uncompressed, high-resolution communication systems were adopted for DtoD conferences/consultations.^31,32 These systems required close collaboration between physicians and technical experts. To institutionalize this interdisciplinary cooperation across the region, the Asia-Pacific Advanced Network Medical Working Group (APAN-MWG) was established in 2005.³³ This working group consists of physicians from Asian countries who regularly use telemedicine in their clinical practice. The members of the working group include network engineers and hospital IT staff who are part of the telemedicine teams. These teams conduct not only one-on-one remote consultations and conferences but also case conferences and live demonstrations between multiple domestic and international hospitals. Their activities range from routine telemedicine consultations to large-scale events such as academic conferences and research meetings.

Recent innovations in virtual reality (VR) and immersive communication have enhanced this mode of communication.^34,35 For instance, 360° videos display images in all directions from the camera’s position, allowing viewers to explore the scene and focus on specific areas.^36,37 These videos are applied in fields such as surgery, trauma, dental radiology, and emergency medicine,^38–41 contributing to improved learner satisfaction and self-efficacy, and supporting better learning outcomes.⁴¹ Another innovative modality is the metaverse,^42–46 which enables users to interact, socialize, work, and learn online through digital avatars in immersive environments. From VR interventions for chronic pain management⁴⁷ to medical training,⁴⁴ the metaverse can be applied in various scenarios.

The expansion of telemedicine has increased interest in education and training. Indeed, telemedicine education occurs in clinical settings and educational institutions in the United States, Brazil, Australia, the United Kingdom, and Japan.⁴⁸ Training programs are embedded in medical school curricula.^49,50 Recently, integrated guidelines for doctor-to-patient video consultation etiquette have been developed, and e-learning programs to enhance health care professionals’ knowledge and confidence in virtual care have been proposed.^51,52 DtoD telemedicine conferences and consultations require implementing and adjusting networks, video and audio equipment, and videoconferencing systems. Within the APAN-MWG, training and Train-the-Trainer programs for IT staff and physicians have been conducted since 2015.^53–55 Training has been provided to enable domestic and international telemedicine conferences and consultations using academic research and commercial networks, using the latest technologies.⁵⁶ Technological advances have enabled doctors and administrative staff to perform tasks previously handled by hospital IT staff.^53,57,58 Increased use of DtoD conferences after the COVID-19 outbreak has created demand for specific technical skills among staff members. Yet, limited attention has been paid to understanding the technical skills required for DtoD telemedicine led by interdisciplinary teams in Asia. Addressing this gap, the current study collaborated with the APAN-MWG to identify the technical skills necessary for DtoD telemedicine in Asia,^55,59,60 to provide technical skills relating to: –

setting up technology to conduct domestic and international DtoD telemedicine;

–

training to install and use various technologies and equipment relevant to different medical subspecialties;

–

familiarizing trainees with recent advanced technologies emerging in the field.

Methods

This study employed the e-Delphi method to solicit individual opinions of experts within the APAN-MWG interdisciplinary team regarding technical skills needed for DtoD telemedicine conferences and consultations, including emerging technologies by distributing an online questionnaire via email.^61–63 First, a team was organized consisting of investigators, an expert panel, and Delphi participants. The investigators included five members from Japan, Indonesia, and India: three telemedicine researchers and two IT directors at medical facilities. The expert panel comprised five members recruited from Malaysia, Japan, Indonesia, and Australia, including four IT directors in health care facilities (three physicians and one technologist) and one medical education specialist affiliated with a telemedicine center. Thirty participants were recruited from the trainers of the APAN-MWG DtoD training program and the Train-the-Trainer program. All participants were engaged in telemedicine, and the majority (80%) were affiliated with university hospitals or nonuniversity hospitals, followed by government agencies (17%). The aggregated opinions were evaluated, refined, and ultimately consensus was reached.^64–66

In Phase 1, an expert panel was established to formulate a list of DtoD-related technical skills. It identified technical themes and formats; both synchronous and asynchronous models were considered. Live demonstrations and 360° video content were targeted for specific medical fields to identify particular skills. Live demonstrations were often conducted at endoscopy and laparoscopic surgery conferences; thus, the study focused on live demonstrations of endoscopy. Meanwhile, 360° video content was often used to present the operating room environment involving staff from multiple professions; the same was set up accordingly. Five themes and formats were identified:

Overview of DtoD telemedicine and videoconferencing basics

Setting up a large venue for hybrid medical conferences

Live medical demonstration: endoscopy demonstration

360° video on-demand transmission of the operating room

Medical case conferences using the metaverse

In Phase 2, the Round 1 questionnaire was administered on December 1, 2023, to assess the listed items with respect to technical skills. Phase 3 involved organizing and conducting a workshop with the participants to further discuss the items identified by the expert panel, held on December 15, 2023. The workshop was conducted in a hybrid format, allowing both on-site and online participation. In total, 27 participants from 24 institutions in 14 countries attended the workshop. During the workshop, experts presented and discussed their implementation experiences and related problems, as well as the technical skills required, for each of the five themes and formats. After the workshop, the revised Round 2 questionnaire was sent to participants for review on January 25, 2024. Respondents were asked to rate the appropriateness of each item on a 1–9 scale (1: Inappropriate; 9: Appropriate). Questionnaire responses were categorized into three levels: 1–3 (inappropriate), 4–6 (unknown), and 7–9 (appropriate). The recommended consensus rate of the Delphi method ranges from 51% to 70%, with 70% indicating a more cautious threshold.⁶⁷ Therefore, responses of 70% or greater were treated as signifying consensus. The investigators revised the list of skills based on all consensus results, comments, and input from the workshop. Consequently, in the analysis, the number and percentage of respondents who answered 7–9 were calculated.

Ethical considerations

This study was conducted according to the principles of the Declaration of Helsinki and was approved by the Ethics Committee of Kyushu University Hospital (No. 23235-00). All participants signed an informed consent statement prior to participation.

Results

Thirty participants took part in the study. Table 1 shows participants’ characteristics. The majority were hospital IT staff. A doctor and a nurse also took part in the study. Over half (57%) were affiliated with university hospitals, followed by nonuniversity hospitals (23%). Five worked in government sectors. The highest number of participants were from Indonesia and Vietnam (7/30, 23%), followed by Nepal (6/30, 20%), and Japan (3/30, 10%). All 30 participants from 11 countries completed the Round 1 questionnaire, and all drafts were evaluated as appropriate, with 46 comments. After the workshop, 10 skills were added such as “Set up a system for one-to-one consultations/meetings (Topic 1),” “Manage multiple audiovisual signals (Topic 2),” and “Obtain approval from the appropriate management department (Topic 4).” Participants also completed the second survey, and all drafts were evaluated as appropriate, with 44 comments. In total, 43 technical skills were identified by consensus among the participants. The list included skills related to conceptual understanding, technical execution, coordination, operational management, and security policy considerations. Skills needed for hybrid medical conferences received the highest scores (Round 1: 94%, Round 2: 91%), while those for 360° video on-demand transmission received the lowest (Round 1: 85%, Round 2: 81%). The results for each of the five topics are presented in detail below.

Table 1.

Characteristics of the Participants

	n (%)
Occupation
IT^a staff	28/30 (93%)
Physician	1/30 (3%)
Nurse	1/30 (3%)
Gender
Male	26/30 (87%)
Female	4/30 (13%)
Country and region
Indonesia, Vietnam	7/30 (23%) each
Nepal	6/30 (20%)
Japan	3/30 (10%)
Bhutan, India, Malaysia, Mongolia, the Philippines, Sri Lanka, Taiwan	1/30 (3%) each
Affiliation
University hospitals	17/30 (57%)
Hospitals not affiliated with a university	7/30 (23%)
Government sector	5/30 (17%)
Company	1/30 (3%)

Information technology.

Topic 1: Overview of DtoD telemedicine and videoconferencing basics

The workshop discussion resulted in additional four topics. In Round 2, all nine items were agreed upon, with an average agreement rate of 89% (Table 2). A hospital IT staff from Nepal noted:

Table 2.

Consensus on Skills: Topic 1—Overview of DtoD Telemedicine and Videoconferencing Basics

Round 1 (n = 30)			Round 2 (n = 30)
Skill	Median	Consensus	Skill	Median	Consensus
1. Explain the concept of DtoD telemedicine	8	83% (25/30)	No change	9	90% (27/30)
2. Explain the technical formats of DtoD telemedicine	8	87% (26/30)	No change	9	87% (26/30)
3. Explain appropriate internet requirements for DtoD telemedicine	8	77% (23/30)	No change	8	87% (26/30)
4. Explain how to handle medical information based on considerations of ethical issues	8	87% (26/30)	No change	9	90% (27/30)
5. Explain the importance of maintaining an updated system	8	77% (23/30)	No change	9	87% (26/30)
—	—	—	6. Set up a system for one-to-one consultations/meetings	9	90% (27/30)
—	—	—	7. Prepare for efficient conferences/consultations	9	90% (27/30)
—	—	—	8. Participate in a session	9	90% (27/30)
—	—	—	9. Share visual information	9	93% (28/30)

In developing countries like Nepal, where internet facilities in rural hospitals are minimal, they have budgetary constraints for procuring dedicated internet services. In addition, technical IT human resources are limited for maintaining the IT system.

Topic 2: Setting up a large venue for hybrid medical conferences

A participant from Malaysia described the challenges of large venue setups:

A large venue setup is very challenging. Thus, higher skills are required for the IT staff to handle this, especially for sound and video.

A nurse from Nepal highlighted infrastructure and power issues:

Most of the things depend on the computer and hospital infrastructure. Dedicated internet connectivity may be required which is costly. In addition to this, frequent power cuts can be another issue in operating telemedicine.

At the workshop, feedback indicated that staff management and controlling multiple audiovisual signals were important aspects of preparing a large venue. Therefore, the theme “Setup local venue from scratch” was revised to “Setup local venue from scratch for a large audience.” Two items, “Organize staffing for a large venue” and “Manage multiple audiovisual signals” were added. In Round 2, all 11 items were agreed upon, with an average agreement rate of 91% (Table 3).

Table 3.

Consensus on Skills: Topic 2—Setting up a Large Venue for Hybrid Medical Conferences

Round 1 (n = 30)			Round 2 (n = 30)
Skill	Median	Consensus	Skill	Median	Consensus
1. Operate equipment
1-1. Use network	9	90% (27/30)	No change	9	93% (28/30)
1-2. Operate videoconferencing system	9	97% (29/30)	No change	9	93% (28/30)
1-3. Operate audio equipment	9	100% (30/30)	No change	9	93% (28/30)
1–4. Operate visual equipment	9	100% (30/30)	No change	9	93% (28/30)
2. Setup local venue
2-1. Setup local venue from scratch	9	93% (28/30)	2-1. Setup local venue from scratch for a large audience	9	87% (26/30)
2-2. Setup local venue using existing equipment	9	93% (28/30)	No change	9	90% (27/30)
—	—	—	2-3. Organize staffing for a large venue	8	90% (27/30)
—	—	—	2-4. Manage multiple audiovisual signals	8	87% (26/30)
3. Maintain the required quality level for telemedicine
3-1. Maintain audio quality	9	93% (28/30)	No change	9	90% (27/30)
3-2. Maintain camera and contents image quality	9	93% (28/30)	No change	9	90% (27/30)
3-3. Maintain network quality	8	87% (26/30)	No change	8	90% (27/30)

Topic 3. Live medical demonstration: endoscopy demonstration

A hospital IT staff member from Bhutan emphasized the importance of skills for extracting and distributing images from medical devices.

Practical sessions are very important to improve capacity to extract endoscopy images, and to able to provide live demonstration.

Workshop participants thought the theme “Handling live demonstrations” was too abstract. They suggested it should be more specific; for example, switching video sources according to the situation (e.g., picture-in-picture) and monitoring the quality of streaming video. Thus, two items were added: “Managing multiple video signals for live demonstrations” and “Monitoring the quality of streaming video.” In Round 2, all nine items were agreed upon, with an average agreement rate of 89% (Table 4).

Table 4.

Consensus on Skills: Topic 3—Live Medical Demonstration: endoscopy Demonstration

Round 1 (n = 30)			Round 2 (n = 30)
Skill	Median	Consensus	Skill	Median	Consensus
1. Coordinate for high-quality live demonstrations	8	93% (28/30)	No change	8.5	90% (27/30)
2. Extract images from endoscope equipment	9	90% (27/30)	2. Extract images from medical devices	9	83% (25/30)
3. Select appropriate audiovisual equipment	9	90% (27/30)	No change	9	90% (27/30)
4. Create a configuration diagram	9	83% (25/30)	4. Create a technical connection diagram	9	83% (25/30)
—	—	—	5. Manage multiple audiovisual signals for live demonstrations	9	87% (26/30)
5. Handle personal and medical information	8	83% (25/30)	No change	8.5	90% (27/30)
6. Organize a pretest	9	97% (29/30)	No change	9	93% (28/30)
—	—	—	8. Monitor streaming audiovisual quality	9	93% (28/30)
7. Handle live demonstrations	9	97% (29/30)	No change	9	90% (27/30)

Topic 4. 360° video on-demand transmission of the operating room

In Round 1, two participants noted little experience using a 360° camera. Additional comments included:

Need more reviews ∼360° image quality such as the real image ratio… (Hospital IT staff, Indonesia)

Ethical clearance and administrative approval-related issues were not included! (Hospital IT staff, Sri Lanka)

In response, “Obtain approval from the appropriate management department” was added to the existing theme “Predict and address security risks, such as inappropriate exposure of patient information and viewer age restrictions.” In Round 2, all eight items were agreed upon, with an average agreement rate of 81% (Table 5).

Table 5.

Consensus on Skills: Topic 4—360° Video on-Demand Transmission of the Operating Room

Round 1 (n = 30)			Round 2 (n = 30)
Skill	Median	Consensus	Skill	Median	Consensus
1. Create a configuration diagram for a 360° video on-demand transmission	8	83% (25/30)	1. Create a technical connection diagram for a 360° video on-demand transmission	8	77% (23/30)
2. Retrieve sufficient quality images from a 360° camera	8	87% (26/30)	No change	8	80% (24/30)
3. Input the demonstrator’s voice with appropriate quality	8	87% (26/30)	3. Input the demonstrator’s voice with high quality	9	83% (25/30)
4. Record and mix 360° videos	8	83% (25/30)	No change	8	77% (23/30)
5. Publish the on-demand 360° video	8	87% (26/30)	No change	8	77% (23/30)
6. Setup head-mount display	7	80% (24/30)	No change	8	77% (23/30)
7. Predict and address security risks, such as inappropriate exposure of patient information and viewer age restrictions	9	90% (27/30)	No change	9	87% (26/30)
—	—	—	8. Obtain approval from the appropriate management department	9	93% (28/30)

Topic 5. Medical case conferences using the metaverse

In Round 1, two participants noted limited experience using the metaverse. The hospital IT staff from Bhutan stated:

Trainees should be exposed to the latest equipment and technologies. They should keep abreast with the technologies, applications, and sound and video systems so that they know the right thing to do when they are exposed to the situation/equipment they are not familiar with… .

The hospital IT staff from Indonesia raised privacy concerns:

Talking about the metaverse means also talking about privacy protection. It needs more study ahead.

At the workshop, participants agreed to add new technical elements despite some having little experience with the metaverse. Skills such as simultaneous distribution (e.g., YouTube) were deemed necessary to enable spectators who had not installed the system to view the sessions. Accordingly, the item “Livestreaming settings” was added. In Round 2, all five items were agreed upon, with an average agreement rate of 84% (Table 6).

Table 6.

Consensus on Skills: Topic 5—Medical Case Conferences Using the Metaverse

Round 1 (n = 30)			Round 2 (n = 30)
Skill	Median	Consensus	Skill	Median	Consensus
1. Prepare the metaverse application	8	77% (23/30)	No change	8	80% (24/30)
2. Create a common workspace	8	83% (25/30)	No change	8	83% (25/30)
3. Share medical contents (surgical movie, CT scanned image, etc.)	8	87% (26/30)	No change	8	87% (26/30)
4. Use annotation	8	83% (25/30)	No change	8	83% (25/30)
—	—	—	5. Setup livestreaming	9	87% (26/30)

Discussion

This study aimed to identify the technical skills needed for conducting DtoD telemedicine conferences/consultations. Thirty technicians and medical staff from medical and government facilities in 11 Asian countries were recruited. Based on the results, 43 technical skills were identified across five categories. Skills for using recent innovations, such as 360° video streaming and the metaverse, were also considered important. Despite our cautious target of 70% for the agreement rate, we exceeded 90% agreement in Round 2 (22 out of 42). The average agreement rate for each category was high, exceeding 84%. The category with the highest agreement rate was Topic 2, “Setting up a large venue for hybrid medical conferences,” which may reflect the greater number of implementation opportunities available for this topic compared with the other four. The lowest agreement rate (81%) occurred for Topic 4, “A 360° video on-demand transmission of the operating room,” likely because it is a relatively new technology. Similarly, Topic 5, “Medical case conferences using the metaverse,” involving another new technology, had an agreement rate of 84%, slightly lower than other items. Comments indicated that further research is needed on privacy issues related to using these technologies. During workshop discussions, participants agreed that these new technologies are being applied in DtoD telemedicine and should be included, but the technical skills acquired may require updating as the technologies become more widely adopted. The opinions of the 30 experts converged, and they agreed on the capabilities identified in this study. Although originally planned to continue until consensus was reached on all items, the final consensus rate of 70% was exceeded for all items in Round 2. In past studies, the Delphi method has often been conducted in two to three rounds.^64,68 After analyzing the second-round results, the final findings were provided to all respondents during the Round 2 group feedback session.

The list of technical skills had the following characteristics. In Topic 1, in addition to understanding the concept of DtoD, “basics of video conferencing” was added in Round 2. The item “Prepare for efficient conferences/consultations,” added in Round 2, is necessary to ensure efficiency, a major benefit of telemedicine.^69,70 To minimize the time commitment of the doctor in charge on the day of the consultation or conference, sharing relevant information in advance with the participants and thoroughly checking the operation of equipment to avoid wasting time is necessary. Regarding Topic 2, focusing on a hybrid setup in a large venue, the operation of video and audio equipment and network devices, quality control, and skills related to management of multiple audiovisual signals and staff—unique to large venues—were identified as important. Hospital IT staff from Nepal repeatedly emphasized the need to address limitations in technical infrastructure and IT staff at hospitals in rural areas. Geographically, Nepal has a large mountainous area and has been implementing telemedicine; thus, the ability to manage communication problems in resource-limited rural areas is essential.^71,72 Accordingly, video, audio, and network maintenance items were included in this topic. Regarding Topic 3, the hospital IT staff from Bhutan highlighted the importance of extracting images from medical devices. Live demonstrations are often used in endoscopic diagnosis, treatment, and surgery, and smoothly switching between images from medical equipment and those held by the demonstrator is important. Extraction and switching skills are required to present the images that the receiver needs. Furthermore, during live demonstrations, distribution personnel were previously satisfied with merely sending video and did not monitor how it appeared online, making monitoring at their own facilities necessary. Topic 4 focused on the distribution of 360° videos. As these videos present content in all directions, important aspects included obtaining permission from the medical facility that owns the videos, predicting security risks, and taking measures to address them. Regarding Topic 5, which involved the metaverse, as with other new systems, it is deemed necessary to prepare for streaming distribution for participants unable to join due to technical constraints. The metaverse is a promising yet nascent technology. While its potential to transform industries is significant, it remains in the early stages of development. Key challenges include high costs of VR and AR devices, limited infrastructure in many regions, and fragmented platforms lacking interoperability.⁴⁴ Public adoption is also uneven, with concerns regarding privacy, data security, and mental health affecting user confidence.⁷³ Despite these hurdles, the metaverse is steadily growing, with companies and developers exploring innovative applications, from virtual classrooms to collaborative workspaces.⁷⁴ Its evolution mirrors that of the internet—initially met with skepticism but gradually becoming integral to daily life. As technology advances and societal adaptation improves, the metaverse can potentially redefine how people connect, create, and collaborate in a digital-first world. Therefore, keeping abreast of the most recent technology through related experiences and training is crucial. In DtoD, it is possible to change discussion partners depending on the topic, explore complex three-dimensional structures such as organs collaboratively, and communicate with others using gestures or maintaining a distance.^43,75,76 These differences in communication formats have the potential to expand and deepen telemedicine activities.

Limitations and strengths

The participants were members of the APAN-MWG, which implements DtoD; therefore, there is a possibility that skills are limited toward technical content. While this study did not restrict activities to specific medical fields, the live demonstration for Topic 3 targeted a case in endoscopy, and the 360-degree video for Topic 4 targeted a case in surgery. Therefore, different skills may be required for activities in other medical fields. This study targeted relatively new technologies such as VR, which were covered in the APAN-MWG training program. However, several participants reported limited experience with these technologies, which may have affected their evaluations. Nonetheless, the study incorporated opinions from participants in 11 Asian countries, including respondents from facilities with limited equipment and human resources, allowing the creation of a highly versatile list. Few prior studies have clarified the technical skills needed for DtoD telemedicine, which this study identified across conditions and contexts, including synchronous and asynchronous models, domestic and international settings, and multiple medical departments. As new technologies continue to be developed and are likely to be introduced in the future, this study can serve as a foundation for improving technical skills needed for DtoD telemedicine.

Conclusions

To our knowledge, this study is the first to identify the technical skills needed for DtoD telemedicine conferences and consultations in the Asian context. In total, 43 technical skills were identified across five categories, including basic knowledge, currently popular modes of livestreaming and large-scale hybrid setups, as well as 360° video streaming and the metaverse. These skills can inform the development of future training programs and evaluation criteria. Building and disseminating such programs will enable DtoD activities to be implemented more efficiently and support further development.

Authors’ Contributions

K.K.: Conceptualization, investigation, formal analysis, and writing—original draft. S.U.: Investigation, formal analysis, and writing—original draft. Y.H.: Investigation, formal analysis, and writing—original draft. S.E.: Investigation and supervision. A.K.: Conceptualization and investigation. T.M.: Conceptualization, investigation, writing—reviewing and editing and supervision.

Footnotes

Acknowledgments

The authors are grateful to the members of the APAN-MWG.

Author Disclosure Statement

None declared.

Funding Information

This work was supported by JSPS KAKENHI [Grant Numbers JP23K25702, JP23K17621, JP24K06306, JP24K06247, 22K02858, 24K06306].

Abbreviations

References

GBD 2015 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990–2015: A systematic analysis for the global burden of disease study 2015. The Lancet, 2016; 388(10053):1545–1602; doi: 10.1016/S0140-6736(16)31678-6

Kosowicz

, Tran

, Khanh

, et al. Lessons for Vietnam on the use of digital technologies to support patient-centered care in low- and middle-income countries in the Asia-Pacific region: Scoping review. J Med Internet Res, 2023; 25:e43224; doi: 10.2196/43224

Catindig

J-AS

, Venketasubramanian

, Ikram

, et al. Epidemiology of dementia in Asia: Insights on prevalence, trends and novel risk factors. J Neurol Sci, 2012; 321(1–2):11–16; doi: 10.1016/j.jns.2012.07.023

Lewin

, Puri

, Quek

, et al. Management of sarcoma in the Asia-Pacific region: Resource-stratified guidelines. Lancet Oncol, 2013; 14(12):e562-70–e570; doi: 10.1016/S1470-2045(13)70475-3

Shen

, Shan

Y-S

, Hu

H-M

, et al. Management of gastric cancer in Asia: Resource-stratified guidelines. Lancet Oncol, 2013; 14(12):e535-47–e547; doi: 10.1016/S1470-2045(13)70436-4

Roy

, Chaudhuri

. Influence of socioeconomic status, wealth and financial empowerment on gender differences in health and healthcare utilization in later life: Evidence from India. Soc Sci Med, 2008; 66(9):1951–1962; doi: 10.1016/j.socscimed.2008.01.015

Clegg

, Reichman

, Miller

, et al. Impact of socioeconomic status on cancer incidence and stage at diagnosis: Selected findings from the surveillance, epidemiology, and end results: National Longitudinal Mortality Study. Cancer Causes Control, 2009; 20(4):417–435; doi: 10.1007/s10552-008-9256-0

Faulks

, Freedman

, Thompson

, et al. The value of education in special care dentistry as a means of reducing inequalities in oral health. Eur J Dent Educ, 2012; 16(4):195–201; doi: 10.1111/j.1600-0579.2012.00736.x

Gamble

, Beech

, Blackshear

, et al. Changes in physical activity and television viewing from pre-pregnancy through postpartum among a socioeconomically disadvantaged perinatal adolescent population. J Pediatr Adolesc Gynecol, 2021; 34(6):832–838; doi: 10.1016/j.jpag.2021.06.009

10.

Nethan

, Hariprasad

. Project ECHO : A potential best-practice tool for training healthcare providers in oral cancer screening and tobacco cessation. J Cancer Educ, 2019

11.

Koenig

, Wenger

, Graham

, et al. Managing professional knowledge boundaries during ECHO telementoring consultations in two Veterans Affairs specialty care liver clinics: A theme-oriented discourse analysis. J Telemed Telecare, 2019; 25(3):181–189; doi: 10.1177/1357633X18756454

12.

Lopez

, Baker

, Milbourne

, et al. Project ECHO: A telementoring program for cervical cancer prevention and treatment in low-resource settings. J Glob Oncol, 2017; 3(5):658–665; doi: 10.1200/JGO.2016.005504

13.

Zhu

, Merchant

KAS

, Mohr

, et al. Real-time learning through telemedicine enhances professional training in rural emergency departments. Telemed J E Health, 2021; 27(4):441–447; doi: 10.1089/tmj.2020.0042

14.

Steeves-Reece

, Elder

, Graham

, et al. Rapid deployment of a statewide COVID-19 ECHO program for frontline clinicians: Early results and lessons learned. J Rural Health, 2021; 37(1):227–230; doi: 10.1111/jrh.12462

15.

Sockalingam

, Kirvan

, Pereira

, et al. The role of tele-education in advancing mental health quality of care: A content analysis of project ECHO recommendations. Telemed J E Health, 2021; 27(8):939–946; doi: 10.1089/tmj.2021.0122

16.

Eisenstein

, Kopacek

, Cavalcante

, et al. Telemedicine: A bridge over knowledge gaps in healthcare. Curr Pediatr Rep, 2020; 8(3):93–98; doi: 10.1007/s40124-020-00221-w

17.

Suppan

, Stuby

, Carrera

, et al. Asynchronous distance learning of the National Institutes of Health Stroke Scale during the COVID-19 pandemic, e-learning versus video: A randomized trial. J Med Internet Res, 2021; 23(1):e23594; doi: 10.2196/23594

18.

Ramnath

, Hill

, Schultz

, et al. An in-person and telemedicine “hybrid” system to improve cross-border critical care in covid-19. Ann Glob Health, 2021; 87(1):1–12; doi: 10.5334/aogh.3108

19.

Ministry of internal affairs and communications. Telemedicine model reference book -Doctor to doctor telemedicine-(in Japanese). 2022. Available from: https://www.soumu.go.jp/main_content/000812534.pdf

20.

Benger

. A review of telemedicine in accident and emergency: The story so far. J Accid Emerg Med, 2000; 17(3):157–164; doi: 10.1136/emj.17.3.157

21.

Lun

. Telemedicine in the Asia Pacific. Int J Med Inform, 2001; 61(2–3):85–86; doi: 10.1016/S1386-5056(01)00129-0

22.

Hjelm

, Hazlett

, Hsieh

, et al. Moving with the sun. Int J Med Inform, 2001; 61(2–3):97–101; doi: 10.1016/S1386-5056(01)00131-9

23.

Nakajima

, Juzoji

, Wijarnpreecha

, et al. The final report of the project “AMINE” The Asia Pacific Medical Information Network using with ETS-V. Int J Med Inform, 2001; 61(2–3):87–96; doi: 10.1016/S1386-5056(01)00130-7

24.

Kasitipradith

. The Ministry of Public Health telemedicine network of Thailand. Int J Med Inform, 2001; 61(2–3):113–116; doi: 10.1016/S1386-5056(01)00133-2

25.

Ferrer-Roca

, Vilarchao-Cavia

, Troyano-Luque

, et al. Virtual sonography through the Internet: Volume compression issues. J Med Internet Res, 2001; 3(2):E21–E60; doi: 10.2196/jmir.3.2.e21

26.

Gandsas

, Pleatman

, Altrud

, et al. Live broadcast of surgery through the internet. Lancet, 1996; 348(9037):1314; doi: 10.1016/S0140-6736(05)65795-9

27.

Kruse

, Beilenhoff

, Axon

, European Society of Gastrointestinal Endoscopy Nurses and Associates. ESGE/ESGENA guideline for live demonstration courses. Endoscopy, 2003; 35(9):781–784; doi: 10.1055/s-2003-41592

28.

Atik

, Gorucu‐Coskuner

, Taner

. The effect of live‐video demonstration on dental students’ orthodontic bending performance. J Dent Educ, 2020; 84(3):377–384; doi: 10.21815/jde.019.172

29.

Fajadet

, Wood

, Wijns

. Live-case demonstrations: Putting patients first. Eur Heart J, 2017; 38(42):3173–3176; doi: 10.1093/eurheartj/ehv657

30.

Shimizu

, Ohtsuka

, Takahata

, et al. Remote transmission of live endoscopy over the Internet: Report from the 87th Congress of the Japan Gastroenterological Endoscopy Society. Dig Endosc, 2016; 28(1):92–97; doi: 10.1111/den.12508

31.

Morita

, Matsuoka

, Fujiki

, et al. Implementation of Distance Classroom Visitations and Distance Evaluation Meetings: A comparison between DVTS and mpeg2ts. Proceedings of the second IASTED International Conference on communications, internet, and information technology 2003; 509–514.

32.

Hahm

, Shimizu

, Nakashima

, et al. Telemedicine with digital video transport system. Korean J Gastroenterol, 2004; 43(6):370–375.

33.

Anonymous. Asia-Pacific Advanced Network, Medical Working Group. 2024. Available from: https://apan.net/elements/working-groups/medical/

34.

Bruening

, Truckenmueller

, Stein

, et al. 360° 3D virtual reality operative video for the training of residents in neurosurgery. Neurosurg Focus, 2022; 53(2):E4; doi: 10.3171/2022.5.FOCUS2261

35.

Cremades Pérez

, Espin Álvarez

, Pardo Aranda

, et al. Augmented reality in hepatobiliary-pancreatic surgery: A technology at your fingertips. Cir Esp (Engl Ed), 2023; 101(5):312–318; doi: 10.1016/j.cireng.2023.02.004

36.

Dede

, Jacobson

, Richards

. Introduction: Virtual, augmented, and mixed realities in education. In: Virtual, Augmented, and Mixed Realities in Education. ( Liu

, Dede

, Huang

., et al. eds). Smart Computing and Intelligence 2017; pp. 1–16; doi: 10.1007/978-981-10-5490-7_1

37.

Mabrook

, Singer

. Virtual reality, 360° video, and journalism studies: Conceptual approaches to immersive technologies. Journal Stud, 2019; 20(14):2096–2112; doi: 10.1080/1461670X.2019.1568203

38.

, Gu

, Bae

, et al. Comparative study of 360° virtual reality and traditional two-dimensional video in nonface-to-face dental radiology classes: Focusing on learning satisfaction and self-efficacy. BMC Med Educ, 2023; 23(1):855; doi: 10.1186/s12909-023-04851-8

39.

Harrington

, Kavanagh

, Wright Ballester

, et al. 360° operative videos: A randomised cross-over study evaluating attentiveness and information retention. J Surg Educ, 2018; 75(4):993–1000; doi: 10.1016/j.jsurg.2017.10.010

40.

Patel

, Hawkins

, Chehab

, et al. Developing virtual reality trauma training experiences using 360-Degree video: Tutorial. J Med Internet Res, 2020; 22(12):e22420; doi: 10.2196/22420

41.

Petrica

, Lungeanu

, Ciuta

, et al. Using 360-degree video for teaching emergency medicine during and beyond the COVID-19 pandemic. Ann Med, 2021; 53(1):1520–1530; doi: 10.1080/07853890.2021.1970219

42.

Hennig-Thurau

, Aliman

, Herting

, et al. Social interactions in the metaverse: Framework, initial evidence, and research roadmap. J of the Acad Mark Sci, 2023; 51(4):889–913; doi: 10.1007/s11747-022-00908-0

43.

Sendra-Portero

, Lorenzo-Álvarez

, Rudolphi-Solero

, et al. The second life Metaverse and its usefulness in medical education after a quarter of a century. J Med Internet Res, 2024; 26:e59005; doi: 10.2196/59005

44.

Lewis

, Popov

, Fatima

. From static web to metaverse: Reinventing medical education in the post-pandemic era. Ann Med, 2024; 56(1):2305694; doi: 10.1080/07853890.2024.2305694

45.

Huang

, Yin

, Lv

, et al. A study on the impact of open source metaverse immersive teaching method on emergency skills training for medical undergraduate students. BMC Med Educ, 2024; 24(1):859; doi: 10.1186/s12909-024-05862-9

46.

Gandi

, Cosenza

, Campetella

, et al. What can the metaverse do for urology? Urologia, 2023; 90(3):454–458; doi: 10.1177/03915603231175940

47.

Ahmadpour

, Randall

, Choksi

, et al. Virtual Reality interventions for acute and chronic pain management. Int J Biochem Cell Biol, 2019; 114(July):105568; doi: 10.1016/j.biocel.2019.105568

48.

Edirippulige

, Armfield

. Education and training to support the use of clinical telehealth: A review of the literature. J Telemed Telecare, 2017; 23(2):273–282; doi: 10.1177/1357633X16632968

49.

Edirippulige

, Brooks

, Carati

, et al. It’s important, but not important enough: EHealth as a curriculum priority in medical education in Australia. J Telemed Telecare, 2018; 24(10):697–702; doi: 10.1177/1357633X18793282

50.

Jumreornvong

, Yang

, Race

, et al. Telemedicine and medical education in the age of COVID-19. Acad Med, 2020; 95(12):1838–1843; doi: 10.1097/ACM.0000000000003711

51.

Pittmann

, Danaher-Garcia

, Ann

, et al. Development and validation of the Telehealth Etiquette Competency checklist : A Delphi study. J Telemed Telecare, 2025; 31(9):1308–1316; doi: 10.1177/1357633X241279494

52.

Chorley

, Azzam

, Chan

. Redesigning continuing professional development: Harnessing design thinking to go from needs assessment to mandate. Perspect Med Educ, 2022; 11(2):121–126; doi: 10.1007/s40037-020-00604-1

53.

Kudo

, Ueda

, Tomimatsu

, et al. Developing evaluation criteria for engineering training programs in remote medical education in Asia. IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE) 2021; 01–05; doi: 10.1109/tale52509.2021.9678723

54.

Kudo

, Moriyama

, Tomimatsu

, et al. A Train-the-Trainers (TtT) program for engineers for hosting multiparty International clinical teleconferences. J Int Soc Telemed eHealth, 2019; 7(Vc):1–8; doi: 10.29086/jisfteh.7.e16

55.

Telemedicine Development Center of Asia. Engineer Training. 2021. Available from: https://www.temdec.med.kyushu-u.ac.jp/eng/seminar/training-asia.html

56.

Kudo

, Ueda

, Hisada

, et al. Training needs analysis on the conducting doctor to doctor telemedicine conference/consultations in Asian medical institutions. Stud Health Technol Inform, 2025; 329:1437–1441; doi: 10.3233/SHTI251076

57.

MacLeod

, Kits

, Mann

, et al. The invisible work of distributed medical education: Exploring the contributions of audiovisual professionals, administrative professionals and faculty teachers. Adv Health Sci Educ Theory Pract, 2017; 22(3):623–638; doi: 10.1007/s10459-016-9695-4

58.

, Rerknimitr

, Kudo

, et al. Telemedicine for gastrointestinal endoscopy: The endoscopic club e-conference in the Asia Pacific region. Endosc Int Open, 2017; 5(4):E244–E252.

59.

Asi@Connect TCC. Telemedicine. 2018. Available from: https://www.tein.asia/sub/?mc=401020

60.

Anonymous. Asia pacific advanced network. n.d. Available from: https://apan.net/

61.

Hasson

, Keeney

, McKenna

. Research guidelines for the Delphi survey technique. J Adv Nurs, 2000; 32(4):1008–1015; doi: 10.1046/j.1365-2648.2000.t01-1-01567.x

62.

Win

. Empirical evidence of internal and external factors influencing users’ motivation toward teleconsultation use. Telemedicine and e-Health, 2023; 00(00):1–16; doi: 10.1089/tmj.2022.0527

63.

Liaw

S-T

, Godinho

. Digital health and capability maturity models—A critical thematic review and conceptual synthesis of the literature. J Am Med Inform Assoc, 2023; 30(2):393–406; doi: 10.1093/jamia/ocac228

64.

Agha

, Fowler

, Saeta

, et al.; SCARE Group. The SCARE Statement: Consensus-based surgical case report guidelines. Int J Surg, 2016; 34:180–186; doi: 10.1016/j.ijsu.2016.08.014

65.

Galpin

, Sikka

, King

, et al. Expert consensus: Telehealth skills for health care professionals. Telemed J E Health, 2021; 27(7):820–824; doi: 10.1089/tmj.2020.0420

66.

Fink

, Kosecoff

, Chassin

, et al. Consensus methods: Characteristics and guidelines for use. Am J Public Health, 1984; 74(9):979–983; doi: 10.2105/AJPH.74.9.979

67.

Polit

, Beck

. Nursing Research: Generating and Assessing Evidence for Nursing Practice. Wolters Kluwer Health/lippincott Williams\& Wilkins; 2008.

68.

Barrios

, Guilera

, Nuño

, et al. Consensus in the delphi method: What makes a decision change? Technol Forecast Soc Change, 2021; 163(October 2020):120484; doi: 10.1016/j.techfore.2020.120484

69.

Shigekawa

, Fix

, Corbett

, et al. The current state of telehealth evidence: A rapid review. Health Aff (Millwood), 2018; 37(12):1975–1982; doi: 10.1377/hlthaff.2018.05132

70.

Wallut

, Peyron

, Hervieu-Bègue

, et al. Efficiency of telemedicine for acute stroke: A cost-effectiveness analysis from a French pilot study. Int J Technol Assess Health Care, 2020; 36(2):126–132; doi: 10.1017/S0266462320000057

71.

Parajuli

, Doneys

. Exploring the role of telemedicine in improving access to healthcare services by women and girls in rural Nepal. Telemat Inform, 2017; 34(7):1166–1176; doi: 10.1016/j.tele.2017.05.006

72.

Peeler

, Dhakhwa

, Mian

, et al. Telemedicine for corneal disease in rural Nepal. J Telemed Telecare, 2014; 20(5):263–266; doi: 10.1177/1357633X14537769

73.

Ruiz Mejia

, Rawat

. Recent advances in a medical domain metaverse: Status, challenges, and perspective. In: 2022 Thirteenth International Conference on Ubiquitous and Future Networks (ICUFN), 2022; pp. 357–362; doi: 10.1109/ICUFN55119.2022.9829645

74.

Suh

, McKinney

, Siu

K-C

. Current perspective of metaverse application in medical education, research and patient care. Virtual Worlds, 2023; 2(2):115–128; doi: 10.3390/virtualworlds2020007

75.

Kye

, Han

, Kim

, et al. Educational applications of metaverse: Possibilities and limitations. J Educ Eval Health Prof, 2021; 18:32–13; doi: 10.3352/jeehp.2021.18.32

76.

Matwala

, Shakir

, Bhan

, et al. The surgical metaverse. Cir Esp (Engl Ed), 2024; 102 (Suppl 1):S61–S65; doi: 10.1016/j.cireng.2023.11.009