Use of Project ECHO Telementoring Model in Continuing Medical Education for Pediatricians in Vietnam: Preliminary Results

Abstract

The ECHO (Extension for Community Health Outcomes) model has been introduced and implemented in several hospitals and health programs in Vietnam since 2015. In 2018, Vietnam National Children’s Hospital (VNCH) officially implemented the ECHO model to provide continuing medical education (CME) credits on pediatrics topics for medical staff in its satellite hospitals and health centers in the Northern region of Vietnam. This paper presents preliminary results of the ECHO program at VNCH. Methods included pre- and post-program assessments of pediatricians’ clinical knowledge, self-efficacy, and professional satisfaction. The analysis compared the differences between pre/post scores descriptively. Knowledge of participants increased by 22.5% points on average. More than 90% of Project ECHO-Pediatrics participants experienced increased confidence. Overall, there was an improvement in participants’ self-efficacy in the post-training compared to the pre-training (range 14.7%-22.6% difference from pre-training). All participants improved on their results in the clinical test immediately after the training and maintained it after 3 months. The study demonstrated the ability Project ECHO to improve healthcare worker knowledge and satisfaction.

Keywords

Project ECHO extension community healthcare outcomes continuing medical education pediatrician Vietnam

Background

Approximately 90% of current hospitals in Vietnam are public hospitals. All of them fall under the Ministry of Health’s direct management, including medical services, human resources, and finance. Projects “satellite hospitals” from 2016 to 2020 and “National telehealth program” from 2020 to 2025 in which central-level hospitals are responsible for training and technology transfer to provincial-level hospitals, who then will continue to train district and commune-level hospitals in the fields of Internal Medicine, Surgery, Obstetrics, Pediatrics, Cardiology, and Oncology.

The Vietnam National Children’s Hospital (VNCH) is one of the leading pediatric hospitals in Vietnam. As a central-level hospital, it is responsible for providing mentorship and training for 18 provincial-level hospitals in 29 provinces as assigned by the Ministry of Health of Vietnam. The training courses are typically organized at VNCH’s facilities in Hanoi. Health professionals would travel annually to participate in these CME-credited courses, which could last from a few days to a few months, depending on the specialty. Although face-to-face training provides valuable experience and knowledge for the participants, it also incurs costs to the trainees and their respective hospitals; and causes a lack of health staff at the provincial-level hospitals.

The Extension for Community Healthcare Outcomes (Project ECHO) model is based on an innovative framework that leverages technology to provide training and ongoing mentorship to community-based primary care providers (PCPs).¹ Project ECHO was initially developed in 2003 to improve healthcare for patients with hepatitis C by training and mentoring PCPs in rural regions using videoconferencing technology.^1,2 As of 2020, project ECHO has been replicated in 49 countries throughout 6 continents, with 528 hubs and 955 different programs. The programs cover a variety of disease conditions, such as rheumatology, dermatology, musculoskeletal disease, asthma, chronic pain, palliative care, inflammatory bowel disease, women’s health, complex regional pain syndrome, dementia, diabetes, epilepsy, hypertension, liver disease, mental health, and transgender care, HIV, geriatric, COVID-19, Autism, perinatal care, managing pneumoconiosis.^2-23

From January 2020 to June 2020, VNCH organized 3 ECHO programs, providing training and mentorship in best-practice care for 3 specialties: pediatric cardiology, pediatric respiratory, and pediatric neonatology. ECHO clinics for each of these specialties were held for 2.5 hours, 4 to 5 days per week, for 2 consecutive weeks. Eighteen provincial-level hospitals registered to send participants to the 3 programs. During each clinic, all pediatric-care provider participants (“spokes”) were connected with each other and to a team of experts (“hub”) through secure multipoint videoconferencing technology (Zoom web-based platform). Members of the expert panel at the hub had specific expertise in each of the 3 specialties and included pediatric cardiologists, pediatric respirologists, neonatologists, and radiologists.

Each ECHO session also included a de-identified case presented by one of the pediatric-care provider (PCP) participants from the provincial-level hospitals for discussion, creating opportunities for shared learning, peer support, and mentoring. The expert hub team also gave verbal and written feedback and recommendations for each case presented, providing support to PCPs as they maintained responsibility for their patient’s care. Resources, including the de-identified case report and other documents, were shared and stored via email or a web-based platform.^24,25

This study aimed to evaluate the initial results of Project ECHO’s effect on PCPs’ satisfaction, clinical knowledge, and self-efficacy in pediatric care.

Methods

Study setting

The study was conducted at the Vietnam National Children’s Hospital and its satellite hospitals from January 2020 to June 2020.

In order to implement Project ECHO, VNCH has prepared human resources for implementation and set up the implementation process in addition to equipping an online software system. This key resource is called the implementation team from the Training and Direction Center. The implementation team is trained regularly following ECHO Institute agenda training from New Mexico University (US). The team component included: 1 program manager, 1 facilitator, 1 coordinator, 1 IT excutive.²⁵ In VNCH, a full ECHO program included 4 phases including phase I: ECHO recruitment; phase II: Prior to ECHO training course; phase III: at ECHO training course; phase IV: Post ECHO training course (please see Table 1).

Table 1.

Implementation process at the ECHO-VNCH.

Phase I	ECHO recruitment	Monitored by
1	Implement survey for provincial-level hospital’s training needs	Implementation team at hub
2	Recruit medical leads (experts) from VNCH’s human resource	Hospital leadership’s decision at hub
3	Based on needs, devise a plan for training and inform the provinces about the agenda training	Implementation team at hub
4	Provincial hospitals submit a list of participants to the course	Implementation team at spokes
Phase II:	Prior to ECHO training course
1	Make consensus with medical leads on agenda training following curriculum, knowledge test (pre-test, post-test. . .), toolkit, supervision activity. . .	Program manager or/and facilitator
2	Organize a Zoom call to go through all regulations during the course	Coordinator and IT executive
3	Organize an online pre-test: evaluate professional knowledge	Coordinator and IT executive
4	Guide case-based presentation following format: Case-presentation at the beginning if participants are not willing to present cases, the VNCH will take a case at their site to present, then share the format with the provinces. They later will be required to submit a clinical case to get used to the format	Facilitator
Phase III	at ECHO training course
1	Didactic: took 30′-45′ by medical lead	Facilitator
2	Case-presentation: 30′-45′ by participants from hub and spoke assigned	Facilitator
3	Q&A discussion, interaction: 30′-45′ by participants and medical lead	Facilitator
4	Assessment on spoke’s case report presentation by medical lead	Facilitator
Phase IV	Post ECHO training course
1	Post-test: assessment on participants’ knowledge after the course	Coordinator and IT excecutive
2	Toolkit to evaluate the course: surveys (including self-efficacy assessment, participant satisfaction), interview guides, supervision reports. . .	Program manager and/or facilitator and medical lead
3	Surpervision activity at spoke sites: trips to the provincial-level hospitals for supervision by VNCH team	Supervision team (program manager and/or facilitator, medical lead)

The ECHO curriculum focused on understanding the diagnosis, treatment, and management of patients in the selected specialties and providing evidence-based care. The content was based on the currently available evidence and best-practice approaches for pediatric cardiology, respiratory, and neonatology (see Supplemental Appendix 1).

Clinical knowledge related to pediatrics (neonatology, respiratory, cardiology) was assessed through the pre- and post-tests and post-3 month test with multiple-choice questionnaires covering clinical analysis and reasoning, diagnosis, treatment approach, prescription, prognosis and risk management, and patient counseling knowledge. All participants applied their capacity to manage patients at their workplace during and right after the practicum period. Pre- and post-ECHO training surveys of self-efficacy and confidence of professional satisfaction were calculated for each PCP by summing scores for each statement, using Likert scale with 1 = Strongly Disagree; 2 = Disagree; 3 = Neither Agree nor Disagree; 4 = Agree, and 5 = Strongly Agree.

Questionnaires and scales were completed using a web-based data collection system using an online survey form designed using © Google Forms.

In this study, we evaluate the outcome of ECHO program in VNCH from level 2 (participant’s satisfaction) to level 4 (participant’s competence) following Moore’s expanded framework.²⁶ Toolkits including questionnaires, scales designed to evaluate each levels (\ see Table 2).

Table 2.

Outcome of the ECHO-VNCH to fulfill Moore’s Levels of Expanded Framework.

Levels of expanded framework	Description of level	Outcome evaluation
Level 1: participation	Number of health professionals who participated in CME activity	Attendance records are well kept in 2020. 201 clinicians fulfilled ECHO-Pediatrics and received 12 462 education hours, similar to 24 924 credits
Level 2: satisfaction	Degree to which expectations of participants about CME activity were met	Questionnaires are completed by every provincial pediatrician after a CME activity. Surveys were conducted via an online questionnaire to evaluate the levels of satisfaction of PCPs with acquired knowledge, class organization, connectivity during the course using the 5-point Likert scale from “1 = completely disagree” to “5 = completely agree” (in Supplemental Appendix 2)
Level 3: Learning: knowledge	Participant statements about what the CME intended them to know	Knowledge tests included a pre-test and a post-test, as well as a post-3 month test that evaluate how well they change their knowledge of pediatrics through process of education via Tele-ECHO (in Supplemental Appendices 5a-c)
Level 4: Learning: competence	The degree to which participants show in an educational setting how to do what the CME activity intended them to be able to do	During each tele-ECHO clinic, provincial pediatricians presented patient cases selected from their facility. They identified the questions they wanted the specialists (medical lead) and peers to consider about the patient in writing and in their presentation in CBD. After each patient case, a summary of recommendations from the specialists and peers was presented on CBD (in Supplemental Appendix 3) will share to presenter. The participants also provided recommendations and guidance to their peers during the teleECHO session in response to a peer’s patient case or during the discussion that followed the case presentation. Surveys about competence and professional satisfaction were conducted via an online questionnaire using the 5-point Likert scale with 1 = very low and 5 = excellent (in Supplemental Appendices 4a and b).

Sample and study design

All pediatric-care providers who participated in the program during the period from January to June, 2020 were included in the sample.

A method evaluation comprised quantitative study including participant’s satisfaction, tests of clinical knowledge, and survey of self-efficacy on the participant’s ability in professional competency and confidence in job satisfaction through their experience at pre- and post-ECHO training. Participants completed questionnaires including tests and surveys as mentioned above at pre-ECHO training (before attending the first ECHO session) and post-ECHO training (after completing the program). Pre- and post-data were linked using participant-generated ID codes.

Data analysis

Descriptive statistics were used to describe and summarize participant characteristics. Mann-Whitney U tests were used to explore differences in pre- and post-ECHO evaluations and P-values reported to indicate the model’s impact on physicians’ clinical knowledge and self-efficacy. Statistical significance was set at P = .05.

Results

As mentioned in the background section, from January 2020 to June 2020, VNCH conducted 3 ECHO programs, providing training and mentorship in best-practice care for 3 specialties: pediatric cardiology, pediatric respiratory, and pediatric neonatology. Through the questionnaires, we looked into multiple aspects such as participants’ clinical analysis and reasoning, diagnosis, treatment approach, prescription, prognosis and risk management, and patient counseling knowledge. The analyses show statistically-significant changes in almost all these aspects among participants’ pre-and post-training. These analyses are presented below.

Two hundred one participants completed both the pre- and post-clinical program-knowledge assessments. The average age of participants was 34.9 years, and 79.6% of participants were female. The average length of professional experience was 15.2 years (range 0-25 years). Most participants were pediatricians (81.6%) (please see Table 3).

Table 3.

Characteristics of the participants of the VNCH—Project ECHO program.

Characteristics	n	Percent
Profession
General Doctor	37	18.4
Pediatrician	164	81.6
Gender
Male	41	20.4
Female	160	79.6
Work experience
1-5 years	11	5.5
6-10 years	54	26.9
10+ years	136	67.7

Analyses examined the participant’s self-efficacy of clinical practice improvement at pre-and post-training. The score of all 6 domains increased by 22.5% points, averagely (please see Table 4).

Table 4.

Self-efficacy assessment at pre-and post-training.

Criteria	Pre-training n = 201	Post-training n = 201	P
Ability to make clinical analysis and reasoning when facing the clinical cases	3.1 (0.75)	3.8 (0.55)	<.001
Ability to make diagnosis based on assessment of patient condition	3.2 (0.66)	3.9 (0.52)	<.001
Ability to follow treatment approach based on VNCH guideline	3.1 (0.67)	3.8 (0.53)	<.001
Ability to make prescriptions based on VNCH guideline	3.1 (0.67)	3.8 (0.56)	<.001
Ability to make prognosis and manage risk related to patient’s disease conditions	3.1 (0.71)	3.8 (0.54)	<.001
Ability to counsel patient related to patient’s disease conditions	3.1 (0.69)	3.8 (0.55)	<.001
Overall	3.1 (0.65)	3.8 (0.49)	<.001

Analyses of the pre/post program revealed that more than 90% of participants experienced increased confidence in their jobs. Overall, there was an improvement in participant self-efficacy in all domains assessed in the post-training compared to the pre-training (range 14.7%-22.6% difference from pre-training) (please see Table 5).

Table 5.

Self-assessment of confidence to meet job demand at pre- and post-training.

Criteria	Pre-training n = 201	Post-training n = 201	P
Confidence to perform professional competency at workplace	3.2 (0.69)	3.8 (0.58)	<.001
Opportunity to share clinical experience with colleagues via telehealth activities	3.2 (0.75)	3.9 (0.57)	<.001
Being professionally supported by experts at hub via telehealth activites	3.3 (0.8)	3.9 (0.57)	<.001
Ability to meet job satisfaction at workplace	3.4 (0.74)	3.9 (0.6)	<.001
Ability to improve capacity of healthcare service delivery at workplace	3.1 (0.71)	3.8 (0.54)	<.001
Overall	3.4 (0.71)	3.9 (0.58)	<.001

Before the program, participants earned an average of 59 points, and most of them were below the cut-off point for a pass (70 points). Immediately after the training, 185 out of 201 completed the test and got an average of 88 points. One-hundred percent of them passed the test. Three months after the training, 139 participants still maintained their knowledge when taking the test (please see Table 6).

Table 6.

Scores obtained by participants at pre-training, post-training, and at 3 months after completing the course.

Time of assessment	Score	P
Before the training (n = 185)	59 (11.3)
After the training (n = 185)	88 (8.2)	<.001
3 months after the training (n = 139)	89 (10.9)	<.001

Not all participants completed the tests.

96.7% of participants were satisfied with the training program using the Project ECHO approach. All aspects, including course content, trainers’ support, training materials, and trainer-participant interaction, were highly appreciated (please see Table 7).

Table 7.

Participants’ satisfaction with the telehealth program.

Criteria	Mean	SD	Min	Max
The lectures are useful	4.37	0.69	1	5
The contents of the lectures are relevant	4.32	0.72	1	5
The lessons are useful/interesting	4.32	0.70	1	5
The clinical cases are useful	4.33	0.71	1	5
The clinical cases are relevant	4.27	0.72	1	5
The clinical cases are helpful/interesting	4.25	0.73	1	5
Trainers use suitable teaching delivery methods	4.29	0.73	1	5
The trainer is highly knowledgeable of the subject	4.47	0.70	1	5
Trainers have good interactions with learners	4.38	0.72	1	5
The training time is reasonable	4.05	0.88	1	5
The training content is updated and relevant for learners	4.34	0.72	1	5
Guidance for preparing devices for online lectures is sufficient	4.36	0.73	1	5
Guidance for setting up and using online tools is sufficient	4.38	0.72	1	5
Provision of training materials is timely and sufficient	4.33	0.74	1	5
Internet connection during the sessions is good	4.20	0.75	1	5
The quality of the images and sound is good	4.15	0.77	1	5
Interaction capacity is good	4.21	0.73	1	5
Satisfaction with the training outcomes	4.15	0.66	2	5

Discussion

Project ECHO-Pediatrics was first implemented in Vietnam in 2020 and has continued to show its suitability in capacity building for health workers at all levels. The ECHO-Pediatrics model that connects the central-level with provincial-level hospitals, also known as the “hub and spoke” model, to promote medical examination and treatment by PCPs at the provincial level, through which patients benefit indirectly. This model’s advantages are notable because the PCPs at the provincial level hospital continue to leverage their knowledge to the district-level and community-level primary healthcare providers. Our evaluation results show that pediatricians’ clinical knowledge is significantly increased. Our findings are consistent with those from other settings^27-33 and again show the potential of the ECHO model in improving knowledge for medical staffs at different levels.

The participant satisfaction survey show that learning goals were achieved through the ECHO training model that reduced feelings of professional isolation, increased in providers’ confidence to manage any aspect of patient care,^29,34,35 tightened clinical network through learning loops, broadened community of medical practice,^15,36,37 and achieved CME.^14,30,38-42

Some studies show Project ECHO has succeeded in building networks to respond to disaster or pandemic like Zika, COVID-19^20,43-51 that provide rapid multidirectional information exchange, disseminating, and sharing information.

A study of mental health and addictions ECHO program in India show an increase in learning and self-confidence among participants.^52,53 Similarly, an evaluation of an ECHO-Oncology program in Africa nations found improved self-confidence and knowledge for participating providers,⁵⁴ as assessed the hepatitis C Project ECHO in New Mexico, which found improved self-efficacy and confidence in professional satisfaction.^1,55 Participants develop confidence and expertise in using evidence-based strategies to support their community.^{18,53,54,56-60}

Project ECHO-Pediatrics incorporates principles, practices, and policies recommended by the Agency for Education and Healthcare Research (AEHR), Agency for Care and Treatment (MOH), and other government panels and professional organizations that solve the problem of health disparities.⁶¹

The ECHO-Pediatrics model at VNCH can only be achieved through the government’s collaboration, especially the MOH, the private sector, and stakeholders in the health care sector. The role in the units of training and coordinating healthcare at provincial hospitals is significant in this process, as they take the leading parts in coordinating among different sectors, as well as monitoring as team implementation in this program. Until now, the ECHO-Pediatrics model at VNCH has been operated using national funding and served 18 satellite hospitals of VNCH under the “Satellite Hospital Project” from 2019 to 2020 and in partnership with the ECHO Institute in virtual platform since 2018. Thus, all courses are appropriately designed, organized, and evaluated according to Decree 22 by the MOH. This rigorous framework, however, could be difficult to maintain when national funding runs out in the future.

Moreover, there is a workforce shortage in implementation of the ECHO program for lower-level healthcare facilities, especially at the district and community-level. Therefore, tele-mentoring programs at present is mostly deployed between central and provincial-level hospitals, while demand remains high at district and community health facilities as well. A plan should be in place to sustain funding for other tele-ECHO activities and proper human resources to run in the future that enable to expand to more underserved locations. Cost-analysis studies are encouraged to find out the effectiveness on cost between ECHO and traditional training models.^62-66

Intrinsic to the value of this paradigm shift in the training and treatment of Pediatric diseases is to enable VNCH, Pediatric network, and MOH to fulfill the tremendous 3-fold mission of health education, clinical care, and research, as well as recognized obligation to protect and improve Pediatrics’ health in community^61,67,68

Through the pilot the model of ECHO-Pediatrics, we have compiled documents of training and guidelines database on clinical practices, scheduled for an online library of pediatric knowledge that will leverage to all pediatric healthcare providers in Vietnam.

Programs such as Project ECHO are ideal venues for the pursuit of clinical research in the field of epidemiology, healthcare service delivery, education conveying best practices, evidence-based medicine, and healthcare cost-effectiveness, which have real-world applicability.^4,36

To sustain and replicate ECHO-Pediatrics to the wider Vietnam healthcare system, primarily to medically underserved areas, we subsequently propose the MOH to continue the funding for this and integrate this model into the “National telehealth project” that will be implemented widely throughout Vietnam from the central to the community level starting in 2021.⁶⁷ Another indication for sustainability and expansion of Project ECHO-Pediatrics would be stable fundings from pharmaceutical companies, hospital’s workforce development funding, and VNCH co-funding mechanisms.

Limitation

This study is subject to several limitations. First, while the evaluation demonstrated an increase in provider’s knowledge and self-efficacy, it did not assess the impact on patient outcomes. However, the Project ECHO model has been shown its effectiveness in improved patient outcomes in other settings and disease areas.^19,28,69,70 Second, though participants experience increased satisfaction, this evaluation did not assess provider retention. Also, clinical performance assessment was limited because the assessment tools cannot detect detailed knowledge or participants’ specific skills. Small sample sizes also did not allow us to conduct further analyses in participant subgroups. The lack of a control group is another study limitation.

Conclusions

The Project ECHO-Pediatrics demonstrated the feasibility of adapting the Project ECHO model to continuing medical education in low- and middle-income settings. The Project ECHO-Pediatrics model has successfully promoted distance learning through virtual communities of practice and increased physicians’ self-efficacy. By helping achieve the goal of delivering the proper knowledge to the right place at the right time for those who need it most, the Project ECHO-Pediatrics model has proven to be a powerful tool for improving the health workforce’s quality in Vietnam and should be regarded as the innovative training model of CME that can be applied widely in the situation of a global pandemic such as COVID-19.

Supplemental Material

sj-pdf-1-his-10.1177_11786329211036855 – Supplemental material for Use of Project ECHO Telementoring Model in Continuing Medical Education for Pediatricians in Vietnam: Preliminary Results

Supplemental material, sj-pdf-1-his-10.1177_11786329211036855 for Use of Project ECHO Telementoring Model in Continuing Medical Education for Pediatricians in Vietnam: Preliminary Results by Le Hong Nhung, Tran Minh Dien, Nguyen Phuong Lan, Pham Quoc Thanh and Pham Viet Cuong in Health Services Insights

Footnotes

Funding:

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project ECHO-Pediatrics was supported by the grant number 774/QĐ-BYT—satellite hospital project (2016-2020), sponsored Zoom-plus platform by ECHO Institute from 2018 to 2022. Sponsors have not been involved in the design, conduct, management of the project, data analyses, or interpretation of this manuscript. Study approval was obtained from the Institutional Ethical Review Board in Hanoi Public Health University in June 2020 (Ref 261/2020/YTCC-HD3).

Declaration of conflicting interests:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Author Contributions

The Authors LHN, TMD, PVC was responsible for the study design of the manuscript. LHN, NPL, PQT was responsible for data collection. LHN, PVC was accountable for the analysis and interpretation of the data. LHN, NPL, PVC performed a critical review of the final manuscript. All authors contributed to drafting the final manuscript. All authors read and approved the final version of the manuscript.

ORCID iD

Pham Quoc Thanh

Supplemental Material

Supplemental material for this article is available online.

References

Arora

Kalishman

Thornton

, et al. Expanding access to hepatitis C virus treatment–extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology. 2010;52:1124-1133.

Arora

Thornton

Murata

, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. New Engl J Med. 2011;364:2199-2207.

Katzman

Fore

Bhatt

, et al. Evaluation of American Indian health service training in pain management and opioid substance use disorder. Am J Public Health. 2016;106:1427-1429.

Arora

Geppert

CMA

Kalishman

, et al. Academic health center management of chronic diseases through knowledge networks: Project ECHO. Acad Med. 2007;82:154-160.

Boynton

Flynn

Box

, et al. Inflammatory bowel disease education: I can hear the extension for community healthcare outcomes. Inflamm Bowel Dis. 2015;21:1407-1408.

Catic

Mattison Melissa

, et al. Echo-age: A video-consultation program to bring geriatric expertise to long-term care.: b100. Am Geriatr Soc. 2013;61.

Davis

Sauerwein

Morrow

, et al. Air force diabetes center of excellence Project ECHO: successful telemedicine with a global reach. Diabetes. 2015;64(S1):A197.

Deming

Anderson

Dodd

, et al. Project ECHO: a novel model for clinical pharmacists in a multidisciplinary telehealth care network for rural and underserved communities. Pharmacotherapy. 2013;33(10):e249.

Dubin

Flannery

Taenzer

, et al. ECHO Ontario chronic pain & opioid stewardship:. Providing access and building capacity for primary care providers in underserviced, rural, and remote communities. Stud Health Technol Inform. 2015;209:15-22.

10.

Frank

Carey

Fagan

, et al. Evaluation of a telementoring intervention for pain management in the Veterans Health Administration. Pain Med. 2015;16:1090-1100.

11.

Katzman

Comerci

Duhigg

, et al. UNM ECHO-Pain and headache program. Headache. 2012;52(8):887-888.

12.

Kauth

Shipherd

Lindsay

Kirsh

Knapp

Matza

Teleconsultation and training of VHA providers on transgender care: implementation of a multisite hub system. Telemed E Health. 2015;21:1012-1018.

13.

Marr

Neale

. Project ECHO: Bringing palliative care consultation to rural New Mexico through a novel telemedicine format. J Pain Symptom Manage. 2012;43(2):448-449.

14.

Scott

Unruh

Catlin

, et al. Project ECHO: a model for complex, chronic care in the Pacific Northwest region of the United States. J Telemed Telecare. 2012;18:481-484.

15.

Ball

Wilson

Ober

Mchaourab

SCAN-ECHO for pain management: implementing a regional telementoring training for primary care providers. Pain Med. 2018;19:262-268.

16.

Society YMApatAMotAE. Child youth epilepsy (CYE) teleECHO: innovative tele-mentoring clinic for child & youth epilepsy care. 2014.

17.

Masi

Hamlish

Davis

, et al. Using an established telehealth model to train urban care providers on hypertension management. J Clin Hypertens. 2012;14(1):45-50. http://www.ncbi.nlm.nih.gov/pubmed/22235823

18.

Bikinesi

O’Bryan

Roscoe

, et al. Implementation and evaluation of a Project ECHO telementoring program for the Namibian HIV workforce. Hum Resour Health. 2020;18:61.

19.

Catic

Mattison

Bakaev

Morgan

Monti

Lipsitz

ECHO-AGE: an innovative model of geriatric care for long-term care residents with dementia and behavioral issues. J Am Med Dir Assoc. 2014;15:938-942.

20.

Archbald-Pannone

Harris

Albero

Steele

Pannone

Mutter

JB.

COVID-19 collaborative model for an academic hospital and long-term care facilities. J Am Med Dir Assoc. 2020;21:939-942.

21.

Bellesheim

Kizzee

Curran

Sohl

ECHO Autism: integrating maintenance of certification with extension for community healthcare outcomes improves developmental screening. J Dev Behav Pediatr. 2020;41:420-427.

22.

Coulson

Galvin

Navigating perinatal care in western North Carolina. N C Med J. 2020;81:41-44.

23.

Sood

Assad

Jarrell

, et al. A virtual community-of-practice approach by rural stakeholders in managing pneumoconiosis in the USA: a cross-sectional analysis. Rural Remote Health. 2020;20:5784.

24.

Fowler

Katzman

Comerci

, et al. Mock ECHO: a simulation-based medical education method. Teach Learn Med. 2018;30:423-432.

25.

Echo

Immersion ebook. Bristol Myers Squibb Foundation. 2019;211. https://www.bms.com/assets/bms/us/en-us/pdf/Project-Echo–Amplify-Your-Impact.pdf

26.

Moore

Jr Green

Gallis

HA.

Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1-15.

27.

Arora

Thornton

Jenkusky

Parish

Scaletti

JV.

Project ECHO: linking university specialists with rural and prison-based clinicians to improve care for people with chronic hepatitis C in New Mexico. Public Health Rep. 2007;122 Suppl 2:74-77. Suppl 2:74-7.

28.

Zhou

Crawford

Serhal

Kurdyak

Sockalingam

The impact of project ECHO on participant and patient outcomes. Acad Med. 2016;91:1439-1461.

29.

Anderson

Zlateva

Davis

, et al. Improving pain care with project ECHO in community health centers. Pain Med. 2017;18:1882-1889.

30.

Arora

Kalishman

Thornton

, et al. Project ECHO: a telementoring network model for continuing professional development. J Contin Educ Health Prof. 2017;37:239-244.

31.

Beste

Glorioso

, et al. Telemedicine specialty support promotes Hepatitis C treatment by primary Care providers in the Department of Veterans Affairs. Am J Med. 2017;130:432-438.e3.

32.

Komaromy

Ceballos

Zurawski

Bodenheimer

Thom

Arora

Extension for community healthcare outcomes (ECHO): a new model for community health worker training and support. J Public Health Policy. 2018;39:203-216.

33.

Glass

Waljee

McCurdy

Sales

Specialty care access network-extension of community healthcare outcomes model program for liver disease improves specialty care access. Dig Dis Sci. 2017;62:3344-3349.

34.

Wood

Unruh

Martinez-Paz

, et al. Impact of a telehealth program that delivers remote consultation and longitudinal mentorship to community HIV providers. Open Forum Infect Dis. 2016;3:ofw123.

35.

Ní Cheallaigh

O’Leary

Keating

, et al. Telementoring with project ECHO: a pilot study in Europe. BMJ Innov. 2017;3:144-151.

36.

Carlin

Zhao

Dubin

Taenzer

Sidrak

Furlan

Project ECHO telementoring intervention for managing chronic pain in primary care: insights from a qualitative study. Pain Med. 2018;19:1140-1146.

37.

Fisher

Hasselberg

Conwell

, et al. Telementoring primary care clinicians to improve geriatric mental health care. Popul Health Manag. 2017;20:342-347.

38.

Katzman

Comerci

Landen

, et al. Rules and values: a coordinated regulatory and educational approach to the public health crises of chronic pain and addiction. Am J Public Health. 2014;104:1356-1362.

39.

Mazurek

Brown

Curran

Sohl

ECHO Autism. Clin Pediatr. 2017;56:247-256.

40.

Komaromy

Duhigg

Metcalf

, et al. Project ECHO (extension for community healthcare outcomes): a new model for educating primary care providers about treatment of substance use disorders. Subst Abuse. 2016;37:20-24.

41.

Shelley

Katzman

Comerci

Jr , et al. ECHO pain curriculum: balancing mandated continuing education with the needs of rural health care practitioners. J Contin Educ Health Prof. 2017;37:190-194.

42.

Warne-Griggs

Brandt

Hoffman

Greever-Rice

Mutrux

ER.

Building regional expertise: perspectives from show-Me ECHO participants. Mo Med. 2020;117:222-227.

43.

Calo

Murray

Francis

Bermudez

Kraschnewski

Reaching the Hispanic community about COVID-19 through existing chronic disease prevention programs. Prev Chronic Dis. 2020;17:E49.

44.

Ganesh

Sahu

Nair

Chand

A smartphone based e-consult in addiction medicine: an initiative in COVID lockdown. Asian J Psychiatr. 2020;51:102120.

45.

Lewiecki

Rothman

MS.

COVID-19, medical education, and bone health: insights from Project ECHO. J Clin Densitom. 2020;23:338-339.

46.

Serper

Cubell

Deleener

, et al. Telemedicine in liver disease and beyond: Can the COVID-19 Crisis Lead to action? Hepatology. 2020;72:723-728.

47.

Sheth

Ganesh

Nagendra

, et al. Development of a mobile responsive online learning module on psychosocial and mental health issues related to COVID 19. Asian J Psychiatr. 2020;54:102248.

48.

Sockalingam

Clarkin

Serhal

Pereira

Crawford

Responding to health care professionals’ mental health needs during COVID-19 through the rapid implementation of project ECHO. J Contin Educ Health Prof. 2020;40:211-214. ePub before print.

49.

Steeves-Reece

Elder

Graham

, et al. Rapid deployment of a statewide COVID-19 ECHO program for frontline clinicians: early results and lessons learned. J Rural Health. 2021;37:227-230.

50.

Wijesooriya

Mishra

Brand

PLP

Rubin

BK.

COVID-19 and telehealth, education, and research adaptations. Paediatr Respir Rev. 2020;35:38-42.

51.

Zweig

patient-doctor telemedicine: virtual care in the era of COVID-19 and beyond. Mo Med. 2020;117:175-176.

52.

Chand

Murthy

Gupta

, et al. Technology enhanced learning in addiction mental health: developing a virtual knowledge network: NIMHANS ECHO. 2014 IEEE Sixth International Conference on Technology for Education, Amritapuri, India. IEEE; 2014;4.

53.

Mehrotra

Chand

Bandawar

, et al. Effectiveness of NIMHANS ECHO blended tele-mentoring model on integrated mental health and addiction for counsellors in rural and underserved districts of Chhattisgarh, India. Asian J Psychiatr. 2018;36:123-127.

54.

Yennurajalingam

Amos

Jr Weru

, et al. Extension for community healthcare outcomes-palliative care in Africa program: improving access to quality palliative care. J Glob Oncol. 2019;5:1-8. doi: 10.1200/JGO.19.00128

55.

Furlan

Zhao

Voth

, et al. Evaluation of an innovative tele-education intervention in chronic pain management for primary care clinicians practicing in underserved areas. J Telemed Telecare. 2019;25:484-492.

56.

Lewiecki

Bilezikian

Bukata

, et al. Proceedings of the 2016 Santa Fe bone Symposium: new concepts in the management of osteoporosis and metabolic bone diseases. J Clin Densitom. 2017;20:134-152.

57.

Lopez

Baker

Milbourne

, et al. Project ECHO: a telementoring program for cervical cancer prevention and treatment in low-resource settings. J Glob Oncol. 2017;3:658-665.

58.

Chhatwal

Chen

Wang

, et al. Assessment of the feasibility and Cost of Hepatitis C elimination in Pakistan. JAMA Netw Open. 2019;2:e193613.

59.

Fokom Domgue

Baker

Manjuh

Lopez

Welty

Schmeler

. Connecting frontline providers in Africa with distant experts to improve patients’ outcomes through Project ECHO: a successful experience in Cameroon. Int J Gynecol Cancer. 2019;29:1446-1447.

60.

Dreizler

Wanjiku

. Tele-ECHO for point-of-care ultrasound in Rural Kenya: a feasibility study. R I Med J (2013). 2019;102(7):28-31.

61.

Decission 774/QĐ-BYT on the Satellite Hospital in Pediatric Care 2016-2020, 2016.

62.

Babineau

Zhao

Dubin

Taenzer

Flannery

Furlan

AD.

The embedded librarian in a telehealth continuing medical education program. J Hosp Librariansh. 2018;18:1-14.

63.

Píriz Alvarez

. Technology for improving accessibility of end-of-life care: extension for community healthcare outcomes project. Curr Opin Support Palliat Care. 2018;12:466-471.

64.

Sagi

Chand

, et al. Innovative telementoring for addiction management for remote primary care physicians: a feasibility study. Indian J Psychiatry. 2018;60:461-466.

65.

Serhal

Arena

Sockalingam

Mohri

Crawford

Adapting the Consolidated Framework for implementation research to create organizational readiness and implementation tools for project ECHO. J Contin Educ Health Prof. 2018;38:145-151.

66.

Faherty

Rose

Chappel

Taplin

Martineau

Fischer

SH.

Assessing and expanding the evidence base for project ECHO and ECHO-Like models: findings of a technical expert panel. J Gen Intern Med. 2020;35:899-902.

67.

Decision 2628/QĐ-BYT: Approval of Tele-Health Proposal, 2020.

68.

Vietnam pediatric healthcare system: achievements and challenges. National Pediatric Conference; 2019; Vietnam National Chilldren’s Hospital.

69.

Mitruka

Thornton

Cusick

, et al. Expanding primary care capacity to treat hepatitis C virus infection through an evidence-based care model–Arizona and Utah, 2012-2014. MMWR Morb Mortal Wkly Rep. 2014;63:393-398.

70.

Arora

Kalishman

Thornton

, et al. Project ECHO (Project Extension for Community Healthcare Outcomes): a national and global model for continuing professional development. J Contin Educ Health Prof. 2018;36:S4X S49.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.90 MB