An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge,Confidence,and Attitudes

Abstract

OBJECTIVES

The goal of this implementation study was to describe the implementation and evaluation of the impact of an online pediatric palliative care training program in Nepal, using the Project ECHO model.

METHODS

The study used mixed methods, including a program case study describing the online learning program and before-and-after surveys of program participants, assessing learning through changes in knowledge, comfort, and attitudes. An end-of-program survey was used to evaluate participants’ overall experiences with the learning program and use of the learning resources.

RESULTS

A literature review, stakeholder surveys, and expert input informed the design of the intervention. The course used the Project ECHO model of online education, with modifications based on the leadership team's previous ECHO experiences and local stakeholder input. The intervention occurred over 9 months, with 22 online teaching sessions. Each session consisted of a didactic lecture, case presentation, and interactive discussion with expert clinical teachers. Fifty-five clinicians in Nepal participated, including physicians (47%), nurses (44%), and psychotherapists (5%). Clinicians reported improvements in knowledge, skills, and attitudes after program participation. Program acceptability scores were high, with 93% of participants reporting that the course provided effective learning.

CONCLUSIONS

Project ECHO can be successfully implemented to deliver continuing professional development in Nepal. Delivering palliative care education online using the Project ECHO model, leads to improved knowledge, skills, and attitudes for clinicians. Project ECHO suggests an innovative solution which can provide training and support to clinicians in settings where educational opportunities in palliative care are limited.

Keywords

Medical education low- and middle-income countries virtual learning end-of-life care online learning palliative care implementation study before-and-after study

Introduction

Children's palliative care focusses on preventing and relieving suffering for children with cancer and other serious or life-limiting illnesses by providing physical, psychological, social, and spiritual support to children and their families. As the global burden of non-communicable diseases continues to increase, the need for children's palliative care in low- and middle-income countries (LMICs) will rise.¹ Already, more than 97% of all children requiring palliative care reside in LMICs, and most cannot access palliative care, due to lack of clinical care programs in these countries.^2,3 Despite global recognition that palliative care is important for children with cancer and other life-limiting conditions, there are very few initiatives which have been developed to address these huge gaps globally.^3–5

Nepal is a lower-middle income country, where an estimated 43,000 children require palliative care annually, however, there are currently no specialized children's palliative care services available.^1,3,6 Without access to palliative care, children and their families are unable to access appropriate medications, supplies, and support from trained healthcare providers (HCP), leaving them without the essential supports they need to relieve suffering in the face of progressive or life-threatening illness.⁷

The lack of children's palliative care programs in Nepal means that there are very few clinical experts who can teach the subject, and thus it is very challenging to build the capacity of HCP to provide children's palliative care.^8,9 Without local palliative care services there are few opportunities for clinical training of HCP, leaving HCP unaware of the importance and potential benefits of palliative care for children and without the skills to treat pain, symptoms, and other sources of suffering.^6,10,11

In Nepal, most medical and nursing colleges have not incorporated palliative care, and it is not mandated nationally.¹² Beyond the undergraduate level, there are no children's palliative care specialist training programs, forcing clinicians to travel to India or further away to access training. This creates significant barriers related to cost of travel and absences from clinical duties and family responsibilities. Several short clinical and classroom-based training programs focussed on adult palliative care have been developed in Nepal, however, these programs see very few children and cannot provide adequate clinical training in children's palliative care.⁶

Online education suggests a potential solution to address the huge shortage of palliative care clinicians by enabling training to reach HCP, even those in rural or remote areas.^13,14 Project ECHO (Extension for Community Healthcare Outcomes) is a model of online education which facilitates learning and mentorship between clinicians in underserved areas and clinical experts in well-resourced areas.¹⁵ ECHO employs a “hub-and-spoke” design, where clinical experts at the “hub” connect with HCPs (“spokes”) on a regular basis to provide education through formal teaching and case discussions, as shown in Figure 1.¹⁶

Figure 1.

Illustrative example of hub and spoke design of project ECHO. From “Pediatric Project ECHO®: A Virtual Community of Practice to Improve Palliative Care Knowledge and Self-Efficacy among Interprofessional Health Care Providers,” by C. Lalloo et al., 2021, Journal of Palliative Medicine, 24(7), p. 1239. CC-BY NC 4.0. Reprinted with permission.

Project ECHO suggests an innovative model of online learning which can help to address the gaps in availability of palliative care training for HCP. Furthermore, project ECHO focusses on building communities of clinicians who can receive mentoring, supporting the implementation of new knowledge into clinical practice.^17,18 Despite the proposed benefits of Project ECHO, there is limited evidence describing its implementation in LMICs.^17,19 Two recent studies have described palliative care ECHO programs in India and Bangladesh, reporting on improvements in knowledge, skills and attitudes towards palliative care among participants.^18,20,21 Studying the implementation and impacts of online training in Nepal is particularly important, to ensure that training meets the local educational, resource, and cultural learning needs of participants. These studies are also important to guide teachers in understanding how Project ECHO and other forms of online education can overcome barriers to accessing education (eg cost of travel and take time off to attend training) and support learning for participants.

Despite the proposed benefits of online education, several studies identify that the lack of face-to-face interaction can be challenging for learners.^13,14 Skilled teachers, who create opportunities for discussion and participation among learners may reduce the impact of online-only interactions, with ECHO participants in India and Bangladesh identifying the importance of teachers in maintaining a learning environment where they felt safe to share their ideas and experiences.²⁰ The aim of this implementation study was to describe the process of development and implementation of an online palliative care training program in Nepal, designed to enhance HCP knowledge and skills. The study addressed the following research questions:

What was the context of the intervention (ECHO program), including factors influencing its implementation?

What were the modifications to the original ECHO model and the rationale for these modifications?

What was the effect of the training program on learners’ knowledge, skills, and attitudes towards palliative care?

What were the facilitators and barriers to optimal learning and participation?

Methods

This was a descriptive implementation study using mixed methods (qualitative and quantitative). The study incorporates a narrative case study and before-and-after surveys of program participants describing the intervention (an online palliative care training program), following the Educational Design Research cycle of Analysis and Exploration; Design and Construction; Evaluation and Reflection.²² A narrative case study is used in implementation research to identify and describe the phenomenon including the context of the implementation and the factors influencing implementation and explain how the implementation can leads to changes.²³ Quantitative methods included before-and-after surveys of program participants, which measured learning through changes in knowledge, skill, and attitudes before the course and at the time of course completion; and end-of-program surveys which evaluated participants’ overall experiences with the ECHO program and use of course learning resources.

Context

In 2017, the government of Nepal adopted a National Strategy for Palliative Care, which led stakeholders (eg, government, HCP, palliative care organizations) to identify the vast gap in children's palliative care, and local HCP have started to consider how to develop children's palliative care to meet this need. The Nepalese Association of Palliative Care (NapCARE) is the national palliative care organization which focusses on palliative care education and advocacy. NapCARE has a committed leadership and strong interest in developing children's palliative care. Two Worlds Cancer Collaboration (TWCC) is a Canadian non-profit foundation, which has been helping build palliative care activities with local partners in Nepal for more than 20 years, including education via in-person workshops and clinical training. Kanti Children's Hospital (KCH), the national government pediatric referral hospital in Kathmandu, is well positioned to lead the development of palliative care services across the country, with interested HCP and healthcare administrators, and the hospital cares for many children with serious illness who would benefit from palliative care each year. However, KCH staff require training and mentorship to be able to develop palliative care services. Similar education and mentorship is needed, for HCP working in more rural and remote regions, since most children with serious illnesses reside in rural areas (82% of the population is rural) and these children frequently access healthcare at regional secondary- or primary-level health centers.

Target sites/personnel

The targeted “sites” (organizations) involved in designing and leading the implementation of the ECHO program included NapCARE, KCH, TWCC, and the Hyderabad Centre for Palliative Care (HCPC). These organizations brought together Project ECHO implementation experts, children's palliative care subject matter experts, and palliative care clinicians in Nepal. Together with TWCC, HCPC has co-lead the implementation of more than 20 palliative care ECHO programs in South Asia, and the team has identified key ECHO modifications which facilitate and enhance learning in the South Asian context.¹⁸ Target learners included multidisciplinary HCP (nurses, physicians, psychologists, counsellors, dieticians, and other allied health workers) who are caring for children with serious illnesses across Nepal. Children needing palliative care included those with cancer, serious neurological, cardiac, respiratory, renal and other relevant conditions, as described in the literature on palliative care.²⁴

Participant survey

ECHO learners were invited to participate in online surveys: 1) At the time of registration (before); and 2) Immediately upon completion of the ECHO program (after). Baseline demographic data was collected at the time of enrolling in the ECHO program. Learners were eligible to complete follow up surveys, if they attended a least one ECHO session and completed the baseline survey. All ECHO learners who completed the surveys were included in analysis of the results.

Data collection and data collection tools

Participants were sent invitations containing links to the online surveys by email and WhatsApp. Participants were asked about their expectations of Project ECHO, their knowledge and attitudes towards palliative care, and comfort related to performing palliative care skills, with specific questions corresponding to the course learning outcomes. Due to a lack of validated tools for interdisciplinary clinicians, the study team developed a palliative care knowledge, skills, and attitudes questionnaire based on previous experiences assessing other palliative care ECHO programs in South Asia. The survey was piloted (4 participants, 9.3%) and the wording of several attitude questions was modified to better reflect palliative care concepts in Nepal, based on feedback from local palliative care experts and incorporating ideas from Lalloo et al (2020), who measured participants knowledge, self-efficacy and experiences with a pediatric project ECHO in Canada.¹⁶ Participants were also asked about their experiences and satisfaction with the ECHO program, use of additional learning materials (recordings and database materials), as well self-assessment of knowledge, confidence, and attitudes about palliative care in the end-of-program survey. Survey items include multiple choice questions, open text boxes, and 7-point Likert scales (response options ranged from [1] “strongly disagree” to [7] “strongly agree”; as shown in Supplement S2).

Demographic data about education, professional specialization, professional role, and experience, work location, and the number of children needing palliative care seen each month were collected. Participants’ knowledge was assessed in general and for specific topics included in the training, with 7 specific knowledge items including the stem question “for my scope of practice, I have an appropriate level of knowledge about…”. In the skills domain, there were 15 questions with the stem “within my scope of practice, I am confident in my ability to…”. Previous project ECHO programs have found that participants tend to over-estimate their level of knowledge and confidence prior to participating in education, so participants were asked to re-rate their baseline abilities in these areas during follow up surveys. These adjusted baseline responses were used in subsequent analyses. To evaluate the overall program impacts, we asked participants to describe their overall change in clinical management, with a 5-item rating scale from “no change” to “a large improvement”. Several open-ended questions related to changes in clinical practice, how to improve the program and challenges to participation, were included in the end-of-program surveys of participants’ overall experiences.

Statistical analysis

Descriptive statistics were obtained using Microsoft Excel. Mann-Whitney U-tests were performed to evaluate differences between knowledge, skills, and attitude mean scores that participants reported after completion of the ECHO and before starting the ECHO program. An a-priori statistical significance was set to P = .05. Effect Size was calculated using Cohen's D_s, with an effect size 0.8 or more suggesting a large effect. In some cases, Likert scales were reverse coded to improve clarity of results in presentation. Raw open-ended responses were reviewed and analysed by 2 research team members (CM, MD), responses were grouped based on identified commonalities and subsequently sorted into categories.

Ethics statement

The study was approved by the Children's Hospital of Eastern Ontario Research Ethics Board (CHEO 17/201X, date of approval: Dec 15, 2020). All participants provided written informed consent after receiving comprehensive written information about the study. The study was also reviewed by local ECHO program leaders, although formal local approval was not required.

Results

Narrative case study: Pediatric palliative care ECHO for Nepal

Analysis and exploration phase

A leadership team of key local and international palliative care experts and stakeholders was established, with technical and administrative support from a program coordinator. Input from learners was obtained from a convenience sample of 6 potential participants (pediatricians and pediatric nurses) who completed a brief online survey, about palliative care topics most relevant to them and their program structure preferences (timing and duration of sessions). These results, along with input from stakeholders and experts, and a literature review informed the curriculum development process.^18,20,25

Design and construction phase

The Project ECHO structure enhanced learning by encouraging participants to adapt palliative care principles to the cultural context of Nepal thereby enhancing practice change and mitigating the lack of face-to-face contact in online education.^20,26

Modifications to ECHO model

Modifications to the original Project ECHO model included incorporating local, regional, and international experts in the leadership team to ensure the program met the needs of local HCP. Local faculty believed that international experts would increase participants’ enthusiasm for the program.¹⁸ Several palliative medicine residents were included in the leadership team, and gradually transitioned into more senior leadership roles as their skill increased, thereby developing and mentoring new ECHO leaders. Several of the residents have subsequently lead ECHO programs in India and Bangladesh. Other key adaptations are shown in Table 1.

Table 1.

Key adaptions and refinements of the original ECHO model.

Original Project ECHO Step	Modification	Rationale
Form core leadership team with subject matter expertise	Combination of local experts and international experts on leadership team	Increase participant interest in course
Form core leadership team with subject matter expertise	Addition of physician trainees to leadership team, with gradual transition into leadership roles	Facilitates capacity building in leadership roles in online and in-person learning programs
Learning support	Maintain an online resource library and share video links, presentations and other materials after each session	Increase use of learning resource and knowledge dissemination to participants and other beyond course participants Enhance development of COP Provide opportunities for asynchronous learning
Learning support	Social media chat group for discussion between online sessions
Establish program learning outcomes	Short mobile friendly survey to assess participants’ learning needs	Ensures that course addresses learners’ interests and learning needs
Clinical case presentation	A case template which incorporates key components of palliative care approach	Ensures cases are clear and focus on palliative care issues Provides addition opportunity for coaching and mentorship
Clinical case presentation	Case presentation is reviewed by a clinical expert prior to presentation
Experts provide didactic teaching	Faculty discussion about local medication and treatment availability	Ensures content of teaching is relevant to local resources and available medications
Encourage participation from all learners	Facilitator training program to enhance the role of the facilitator in encouraging participation	Encourages active learning and COP development
Encourage participation from all learners	Use chat feature to encourage participation	Provide lower stakes option for participants who are hesitant to speak
Leadership team debriefs after each session	During debriefing the leadership team considers the need for modifications to the session schedule and content to ensure that the learning goals are reached	Ensures learning is relevant to participants and meeting their learning needs

During this phase, the leadership team met regularly every 2–4 weeks to develop and plan the ECHO program, and these meetings included discussion of the availability of medications and specific treatments in Nepal to support international faculty to understand the local healthcare situation. Figure 2 illustrates a timeline of the key preparatory steps.

Figure 2.

Timeline of leadership team activities prior to launching pediatric palliative care ECHO for Nepal.

Learning program participant recruitment

Social media, text messages, telephone, and word of mouth were used to recruit pediatric health care providers caring for children with life-threatening or life-limiting conditions in Nepal. There were no costs or restrictions to participation, nor were any incentives provided. The program was designed to meet learning needs of HCP but was also open to healthcare administrators and trainees. Program registration was completed online. Information about upcoming sessions was shared with participants by email and through a private group chat on social media (WhatsApp). Participants were permitted to join at any time during the program's duration and were encouraged to register to benefit from access to the course materials and communication.

Program structure and format

Fifty-five learners registered for the program and 46 attended a least one session. Based on feedback from participants and the leadership team about the need to balance the time commitment required for the program and other clinical and personal duties, there were 22 weekly learning sessions conducted in two parts (June to Aug 2020 and January-March 2021). The session topics are shown in Table 2. All sessions used Zoom videoconferencing technology.

Table 2.

Teaching topics of pediatric palliative care ECHO sessions.

	Part 1 (June-August 2020)	Part 2 (January-March 2021)
1	Introduction to Project ECHO, course materials and faculty	Review of key components of Pediatric Palliative Care
2	Introduction to Pediatric Palliative Care	Challenging communication situations: breaking bad news to parents and children
3	Pediatric Palliative Care - how it helps in non-oncological conditions	How to support children with grief and bereavement
4	Importance of teamwork and professional roles in Pediatric Palliative Care	Unique considerations when delivering palliative care to adolescents
5	Opioids and pain management in children & infants, opioid availability in Nepal	Building supportive relationships and communication with children
6	Supporting the family and how to involve community-level health services and supports	Pain management & home-based symptom management
7	How to provide psychosocial support for children with serious illnesses	How to procure and deliver opioids, opioid selection, and rotation
8	Role of play and expressive activities for children with serious illness	Symptom management in neurological conditions
9	Management of symptoms other than pain	Compassionate extubation and ethical issues related to discontinuing life-sustaining treatments
10	How to implement and increase access to Pediatric Palliative Care for children in Nepal	Managing issues with feeding and fluids at the end of life
11	-	End of life care in hospital and at home
12	-	Final Session: Group learning and reflections

Weekly sessions followed the structure shown in Table 3. A case presentation template was provided to participants to support clear and succinct case presentations which focussed on key palliative care issues. During sessions written feedback, questions and discussion was encourage both verbally and using the Zoom chat feature, and one faculty member was assigned to moderate the chat and respond to written questions. If relevant, written questions from the chat were read aloud and discussed.

Table 3.

Key sections of a pediatric palliative care ECHO session (adapted from Doherty, Rayala et al (2021)).

Section	Duration	Description	Purpose
Introduction	3-5 min	◼ Facilitator introduces session and welcomes participants	◼ Develop a supportive online COP
Didactic Presentation	20 min	◼ The experts provide didactic teaching session	◼ Provide new knowledge to participants
Discussion	15 min	◼ Facilitator encourages discussion and questions from participants. ◼ Discussion about practical challenges based on local healthcare situation	◼ Reflection and clarification of new knowledge ◼ Application of knowledge to personal clinical practice
Case Presentation	10 min	◼ Presentation of a clinical case by participant, using a structured template. ◼ Identification of 2–3 key questions from the case for discussion	◼ Clinical application of new knowledge and skills ◼ Discussion of issues most relevant to participants
Case Discussion	20 min	◼ Facilitator supports discussion of key questions from the case ◼ Facilitator may call on participants to share relevant experience or advice ◼ Verbal and written questions are encouraged	◼ Encourage deeper learning through application and discussion ◼ Apply knowledge to the local healthcare context
Summary and Conclusion	5 min	◼ Facilitator summarizes key learning points of the session	◼ Support retention of key new knowledge and skills
Post-session	By email and WhatsApp	◼ Key learning points, relevant clinical resources (articles, book chapters, guidelines), and the video recording shared with participants	◼ Provide opportunities for review of learning with tangible learning materials ◼ Encourages sharing of materials with other HCP not involved in ECHO, supports knowledge diffusion

Facilitator

Each session was facilitated by a member of the leadership team and who focussed on creating a warm and welcoming environment. This was based on a previous study which identified creating a supportive learning environment as an important facilitator activity which made participants feel that they could share their ideas.²⁰

Additional asynchronous learning

In addition to the live (synchronous) videoconference sessions, participants were encouraged to interact between sessions through a group social chat on WhatsApp, which was moderated by faculty. Chat discussion included clinical queries, clarification of new practices and knowledge, and sharing of additional learning resources. After each session, the presentation slides, a summary of key learning points, and the private video recording link were shared with participants via email and WhatsApp. Participants were encouraged to review these materials and share them with colleagues outside of the ECHO program as well, since previous programs have identified the benefits of learners using these resources to teach their colleagues and for personal study.²⁰

Participants completed a group project (Supplement S1) at the midpoint of the course, consisting of developing a plan for how to increase access to pediatric palliative care at their health facility. The goal of the project was to support participants to identify the barriers and enablers of palliative care locally and reflect on how best to address the barriers.

Evaluation and reflection phase

Debriefing

Weekly leadership team debriefings after each ECHO session facilitated rapid adaptions of the learning program. Modifications to the style of facilitation were made to increase learner participation during discussions. Debriefing also helped faculty to identify knowledge gaps or misunderstandings which they then addressed in subsequent sessions. Potential harms to participants were also discussed and strategies were developed to mitigate these. There were no significant harms to participants identified.

Learner survey: Participants characteristics

Forty-three (78.1% of registrants) HCP completed the knowledge, skills and attitudes survey before the ECHO program and 27 out of 41 (65.8%) eligible HCP completed the survey at the end of the course. Participant characteristics are shown in Table 4. The average number of participants per session was 33 (range 21-38). The average attendance for individual learners was 14 sessions (range 1-22).

Table 4.

Characteristics of participants for project ECHO on pediatric palliative care in Nepal (n = 43).

	n	%
Profession
Nurse	19	44.2%
Consultant Physician	14	32.6%
Medical Officer or Resident	6	14.0%
Psychotherapist	2	4.7%
Social Worker	1	2.3%
Nutritionist	1	2.3%
Primary Practice Setting
Government Hospital	38	88.4%
Hospice	3	7.0%
Nursing College	2	4.7%
Gender (n = 43)
Female	22	51.2%
Male	15	34.9%
Undisclosed	6	14.0%
Primary Focus of Professional Work ^a
Pediatric Oncology	16	37.2%
Pediatric Palliative Care	9	20.9%
Adult Palliative Care	7	16.3%
Pediatrics	7	16.3%
Child Psychiatry	2	4.7%
Radiation Oncology	1	2.3%
Pediatric Intensive Care	1	2.3%
Anesthesia	1	2.3%
Pediatric Neurology	1	2.3%
Missing data	3	7.0%
Years in clinical practice
<1 year	3	7.0%
1-5 years	16	37.2%
6-10 years	10	23.3%
11-15 years	5	11.6%
>15 years	6	14.0%
Missing data	3	7.0%
Average (SD)	8.3 years (7.1)
Highest level of education completed
Undergraduate	10	23.3%
Post-graduate Training or Specialization	26	60.5%
Masters or PhD	7	16.3%
Number of pediatric palliative care patients managed monthly
0	2	4.7%
1-10	21	48.8%
11-25	7	16.3%
26-50	5	11.6%
>50	5	11.6%
Missing Data	3	7.0%

Multiple responses permitted

Learner survey: Changes in participants’ knowledge, skills, and attitudes

Baseline responses indicated the areas where participants perceived their knowledge to be lacking, as indicated by responses of “strongly disagree”, “disagree” and “somewhat disagree”. These areas included management of end-of-life care (n = 7/26, 26.9%), breaking bad news (n = 6/26, 23.1%), managing pain (n = 6/26, 23.1%), and managing anxiety and depression in children (n = 6/26, 23.1%).

Areas where participants commonly perceived their clinical skills to be lacking included transitioning from a curative to palliative care approach (n = 7/26, 26.9%), serving as a workplace expert in palliative care (n = 7/26, 26.9%), caring for bereaved family members (n = 6/26, 23.1%) and discussing death with parents (n = 6/26, 23.1%).

Participants’ baseline attitudes related to palliative care included a belief that avoidance of talking about death with parents of a dying child was appropriate as 12 of 26 participants (46.1%) agreed with this statement (includes “strongly agree”, “agree”, and “somewhat agree” responses). Nine of 26 respondents (34.6%) also agreed with the statement that “palliative care represents a failure of modern medicine” at baseline. By the end of the course, average participant responses towards palliative care improved in all 8 areas evaluated, statistically significant changes in 6 of the 8 areas. Sub-group analyses using years of professional experience and type of HCP were performed comparing professional and there were no significant differences identified. Table 5 shows the change in mean knowledge, skills, and palliative care attitudes before and after participation.

Table 5.

Changes in Likert scale scores of participants’ knowledge, self-rated confidence, comfort, and attitudes about palliative care at baseline and the end of ECHO program. (n ranged from 25-27 for each individual question, full data set available on request).^a

	Mean, Baseline	Mean, End of program	Mean Difference	Effect Size^b
Knowledge (For my scope of practice, I have/had an appropriate level of knowledge about…)
Medical conditions appropriate for palliative care	4.81	6.27	1.46	1.57
How to introduce palliative care to families	4.86	6.26	1.40	1.48
Managing pain in children	4.58	6.33	1.75	1.64
Managing nausea and vomiting	5.16	6.41	1.25	1.30
Managing depression and anxiety	4.46	5.89	1.43	1.40
Breaking bad news to parents	4.81	6.30	1.49	1.40
Managing end-of-life care	4.50	6.33	1.83	1.74
Confidence and comfort (Within my scope of practice, I am confident in my ability to…)
Identify patients needing palliative care	4.81	6.44	1.63	1.33
Discuss the transition to palliative care with parents	4.50	6.15	1.65	1.55
Discuss the role of palliative care with health care providers	4.88	6.41	1.53	1.54
Support children with serious illness and their families	4.77	6.22	1.45	1.28
Treat pain using morphine in a child	4.69	6.33	1.64	1.46
Manage nausea and vomiting in a child	5.00	6.44	1.44	1.37
Manage shortness of breath (dyspnea) in a child	4.85	6.35	1.50	1.54
Provide bereavement care	4.54	6.30	1.76	1.57
Identify patients who are approaching end of life	4.85	6.30	1.45	1.35
Discuss death with child or their parents	4.38	6.15	1.77	1.61
Discuss stopping treatments which are no longer effective.	4.62	6.27	1.65	1.45
Discuss when NG tube feeding is not appropriate	5.05	6.30	1.25	1.28
Holistically assess patients	4.85	6.41	1.56	1.47
Break bad news to families	4.83	6.25	1.42	1.33
Serve as an expert on palliative care in my workplace	4.27	6.04	1.77	1.33
Attitudes about Palliative Care
Providing palliative care is a worthwhile experience	4.92	6.30	1.38	1.19
Palliative care should be started at the time of diagnosis	4.92	6.56	1.64	1.23
Introducing palliative care for a child with a progressive incurable illness who is expected to live many months is recommended	5.08	6.44	1.36	1.15
Avoiding NG tube feeding at end of life may be good care	4.44	5.62	1.18	0.75
Palliative care requires active care	5.04	6.37	1.33	0.95
When used according to guidelines, opioids do not shorten life expectancy	5.19	6.44	1.25	1.26

Item response options ranged from strongly agree to strongly disagree, (1 = strongly disagree to 7 = strongly agree). Further details of question wording can be found in Data Supplement S2.

Effect Size (Cohen's D_s.). Effects of ≥ 0.8 generally suggest large effect.

Participants provided qualitative responses to questions about changes in their clinical practice since starting the course, reporting how they implemented pain scales (n = 5), used morphine for pain management (n = 4), and encouraged therapeutic play to support emotional coping in children (n = 3). Other responses included more appropriate use of artificial hydration and nutrition at end of life and more skilled counselling of families. One participant also noted that their colleagues were more receptive to initiation of palliative care after these colleagues participated the ECHO program.

Program acceptability and satisfaction

Participants’ course satisfaction was very positive, with the majority indicating that the course was a valuable experience for them (n = 25/27, 92.6%), that they would recommend the course to colleagues (n = 24/27, 88.9%) and that the ECHO structure was an effective way for them to learn (n = 25/27, 92.6%).

The baseline survey evaluated the most common obstacles to participation, which included lack of time (n = 26/37, 70.3%), along with technical issues with Zoom or Internet connection (n = 20/27, 54.1%). No participants indicated that language was a barrier to participation. The end-of-program survey asked participants to identify obstacles preventing them making clinical practice changes and these included: a need for more formal teaching/training courses in pediatric palliative care (n = 12/27, 44.4%), not having enough time to provide the level of care discussed in the ECHO (n = 9/27, 33.3%), and a need for hands-on clinical exposure and mentorship (n = 8/27, 29.6%). Further details related to the perceived benefits and challenges of practice change are shown in Table 6.

Table 6.

Participants’ comfort and learning experience of ECHO program.

Benefits of participation, Participants agreeing with the following statements^a	n	%
This program was an effective way for me to learn	25	92.6%
Participating in this program was a valuable experience for me.	25	92.6%
I would recommend this program to my colleagues.	24	88.9%
I have learned about best practices of care for patients with serious illnesses through this program	24	88.9%
I respect the knowledge of the faculty involved in this program	24	88.9%
This program was a supportive community of practice for me	24	88.9%
This program reduced my professional isolation	23	85.2%
Participant experiences with learning via the ECHO program ^a	n	%
I feel comfortable speaking, asking questions, and sharing my opinion with other participants	20	76.9%
I feel comfortable using videoconferencing to learn	22	81.5%
The faculty are supportive and approachable	25	92.6%
The faculty try to include and engage me in the discussion	23	85.2%
What were the obstacles to changing your clinical practice based on ECHO program? ^b	n	%
I need for more formal teaching or training	12	44.4%
I lack time to provide care as described in ECHO	9	33.3%
I need clinical exposure or hands-on training	8	29.6%
The rest of my team does not provide clinical care in the way it was discussed in ECHO	3	11.1%
Other members of my team are not aware of palliative care and how it can potentially help	3	11.1%
The medications discussed in ECHO are not available in my setting	2	7.4%
The medications discussed in ECHO are too expensive for my patients to afford	1	3.7%
There is no one else at my hospital to help me implement palliative care as discussed in ECHO	1	3.7%

Includes strongly agree, agree, or somewhat agree.

Multiple responses permitted.

Access to learning resources

Most participants were aware of the additional learning resources available to complement the online live sessions (n = 25/27, 92.6%). The most common reasons for reviewing the recordings were to review missed sessions (n = 16/27, 59.2%,) or to review content/learn more (n = 8/27, 29.6%). All participants reported that they reviewed the key learning points for each session, with most doing this all or most of the time (n = 22/27, 81.4%).

Contextual Changes: The program was developed prior to the COVID-19 pandemic and implemented at the beginning of the pandemic. Changes in health priorities may have impacted participants’ and administrators’ motivations during implementation of the ECHO program. Restrictions on in-person gatherings may have impacted readiness to accept online learning.

Discussion

This study describes the implementation and evaluation of an online palliative care course for clinicians in Nepal, a lower-middle-income country. The course content and structure incorporated feedback from key stakeholders, through consultation and a review of the relevant literature. The learning program was led by a collaborative team of national, regional, and international subject matter experts and teachers from Nepal, India, and Canada. The course consisted of 22 online sessions and used the Project ECHO model. Modifications to Project ECHO, based on previous finding and experiences using online learning in palliative care in South Asia. Learners reported significant improvements in their palliative care knowledge, comfort in performing palliative care skills, and attitudes towards palliative care after participating in the course. Most participants were highly satisfied with the program, noting that the course was effective for their learning and that they would recommend it to others.

We observed significant positive impacts on participants’ knowledge and comfort with palliative care skills from participation in the ECHO program. Our findings are similar to a recent ECHO project in Canada, which also describes significant improvements in participants’ perceived knowledge and self-efficacy.¹⁶ While there are some differences in study design (program length, session frequency, timeframe of data collection), the benefits of the Project ECHO model are present in both high and lower-middle-income countries, suggesting that Project ECHO supports education for healthcare professionals from a range of backgrounds and resource levels. Providing palliative care education which is readily available to pediatric HCPs in Nepal is particularly important since there are currently no palliative care services for children available, and training and mentoring local HCP is needed to support new palliative care programs to develop in Nepal. Project ECHO can support palliative care capacity building activities for HCPs in LMICs and is particularly relevant because this model addresses geographical barriers and resource-limitations of face-to-face training.

Online learning programs have been widely discussed as a model which can increase access to education in remote and rural areas, however, to be most effective virtual learning programs must be adapted to the local health care situation and should introduce participants to clinical management techniques that can be implemented in their local healthcare setting.¹³ The development of this course incorporated these key considerations, with an experienced leadership team ensuring that medication and health resources availability considerations were considered and addressed during didactic teaching and case discussions. This is particularly relevant for courses with international speakers, who may otherwise struggle to provide management recommendations which are applicable locally. The study evaluation findings suggest that this program was successful in providing locally appropriate treatment recommendations, with most learners reporting that the medications discussed in the course were available or affordable for their patients. Discussing the challenges of opioid availability is particularly important when conducting palliative care training in Nepal and other LMICs since difficulty in accessing opioids is a frequent barrier to pain control.²⁷ This course successfully incorporated teaching from local opioid policy experts to ensure that considerations of local opioid availability were addressed.

The course incorporated additional modifications to the original Project ECHO model, including involving key stakeholders and residents in the course leadership team and using brief mobile-friendly surveys to assess potential participants’ learning needs during the course development. Other Project ECHO programs in LMICs should consider implementing these modifications, which may help to ensure that online courses are sustainable. We have previously described additional modifications to Project ECHO, which were implemented during this program including the use of tangible learning materials, in the form of video recordings and presentation materials, to facilitate asynchronous learning.¹⁸ The use of familiar social media platforms and local networks to recruit, engage, and seek program feedback from participants, is an important consideration when implementing online learning, which educators should incorporate when planning new ECHO programs.

Internet connectivity in Nepal has increased significantly in recent years, which enabled this course to encounter minimal technical barriers; however, despite the recent improvements in internet connectivity, more than half of participants still mentioned technology as a barrier to participation. Interestingly, 85% of participants also reported watching session recordings online afterwards, suggesting that the video recordings were easily accessible to participants.

A 2019 systematic review comparing online and offline medical education found no evidence that offline learning improves learning outcomes, suggesting that educators should consider incorporating online learning into medical education.²⁸ However, despite the potential benefits of this approach, further studies are needed to compare the quality of learning online compared to in-person, in palliative care and in LMICs. Several participants in this study identified a need for further clinical training in palliative care, suggesting that the impact of online training may be enhanced by pairing it with clinical placements or workplace-based training.

When designing learning programs in LMICs, the instructional language should be carefully considered, as programs in English may be a significant barrier to learning and our study sought to explore the issue of language as a barrier to learning.^13,25 Interestingly, in this program, none of the participants indicated that English language instruction was a barrier to participation. This may be because most participants were nurses and physicians who are more likely to be comfortable learning in English, compared to community health workers or mid-level health workers in rural settings. Potential participants who were not comfortable in English may have chosen not to participate in the program. Our findings differ from our previous study of Project ECHO in India and Bangladesh, where language was identified as a challenge to full participation for some participants and suggest that educational programs in English can be effective when training nurses and physicians in Nepal.²⁰

Study strengths and limitations

This study describes an innovative online synchronous palliative care course in Nepal. Our findings describe learning experiences of a broad range of professionals including both nurses and physicians from a variety of clinical specialties, suggesting some degree of transferability of findings. Only 59% of eligible participants responded to the final survey, which could lead to selection bias, particularly favoring participants who found the program to be effective or were highly engaged with the learning or who did not face significant language barriers or internet connectivity issues. The questionnaire was administered online but required considerably less bandwidth than the online learning program itself. We also used self-report of knowledge and confidence scores which may not reflect participants’ true capabilities or changes to clinical practice. To reduce language barriers to completing the survey, questions were reviewed with local health care professionals and the wording was revised to better reflect key ideas and concepts as understood in Nepal. Future studies should seek to assess the impact of these programs on the experiences of patients and families.

Conclusion

This study's findings demonstrated that Project ECHO can be successfully implemented to deliver continuing professional development in Nepal. Using an experienced collaborative team of national, regional, and international subject matter experts supports implementation. The study described a model of online training which learners rated as highly effective for learning. The program was associated with significant improvements in palliative care knowledge, skills, and attitudes for learners. Educators and policy makers should consider using Project ECHO to address the urgent need for palliative care for children in settings with limited access to pediatric palliative care expertise.

Supplemental Material

sj-docx-1-mde-10.1177_23821205241234541 - Supplemental material for An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge, Confidence, and Attitudes

Supplemental material, sj-docx-1-mde-10.1177_23821205241234541 for An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge, Confidence, and Attitudes by Anisha Lynch-Godrei, Sudhir Sapkota, Jennifer Rowe, Bishnu Dutta Paudel, Garima Aryal and Megan Doherty in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-2-mde-10.1177_23821205241234541 - Supplemental material for An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge, Confidence, and Attitudes

Supplemental material, sj-docx-2-mde-10.1177_23821205241234541 for An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge, Confidence, and Attitudes by Anisha Lynch-Godrei, Sudhir Sapkota, Jennifer Rowe, Bishnu Dutta Paudel, Garima Aryal and Megan Doherty in Journal of Medical Education and Curricular Development

Supplemental Material

sj-docx-3-mde-10.1177_23821205241234541 - Supplemental material for An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge, Confidence, and Attitudes

Supplemental material, sj-docx-3-mde-10.1177_23821205241234541 for An Online Pediatric Palliative Care Education and Mentoring (Project ECHO) in Nepal: A Program Implementation Case Study and Assessment of Changes in Healthcare Providers’ Knowledge, Confidence, and Attitudes by Anisha Lynch-Godrei, Sudhir Sapkota, Jennifer Rowe, Bishnu Dutta Paudel, Garima Aryal and Megan Doherty in Journal of Medical Education and Curricular Development

Footnotes

Acknowledgements

The authors would like to acknowledge the support of Two Worlds Cancer Collaboration Foundation, the Nepalese Association of Palliative Care, the Pain Relief and Palliative Care Society (India), and ECHO India for their support with the implementation of this Project ECHO. The support of Vineela Rapelli and Dr Gayatri Palat in the development and implementation of the Project ECHO were instrumental to facilitating this research.

Author contributions

Conception and Design: MD, ALG, BP GA, JR, SS

Administrative support: GA, JR

Provision of study materials or patients: BP, MD, SS

Collection and assembly of data: MD, ALG, JR, GA

Data analysis and interpretation: ALG, MD, JR, GA

Drafting of the article or revising it for intellectual content: all authors

Final approval of manuscript: all authors

Accountable for all aspects of the work: all authors

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: JR was supported by funding from the 2020 OMSA/AMS Medical Student Education Research Grant.

Ethics statement

The study was approved by the Children's Hospital of Eastern Ontario Research Ethics Board (CHEO 17/201X). All participants provided written informed consent after receiving comprehensive written information about the study.

ORCID iD

Megan Doherty

Supplemental material

Supplemental material for this article is available online.

References

Connor

Downing

Marston

. Estimating the global need for palliative care for children: a cross-sectional analysis. J Pain Symptom Manage. 2017;53(2):171-177.

Worldwide Hospice Palliative Care Alliance. Global atlas of palliative care [internet]. Worldwide Hospice Palliative Care Alliance; 2020 [cited 2021 Jun 28]:120. Available from: http://www.thewhpca.org/resources/global-atlas-on-end-of-life-care

Kenneson

Hughes-Visentin

Wrigley

, et al. Pediatric palliative care program Implementation in LMICs: A systematic review using SWOT analysis. J Pain Symptom Manage. 2023;66(4):338-350.e11.

Knaul

Farmer

Krakauer

, et al. Alleviating the access abyss in palliative care and pain relief—an imperative of universal health coverage: the lancet commission report. Lancet. 2018 Apr 7;391(10128):1391-1454.

Israels

Renner

Hendricks

Hesseling

Howard

Molyneux

. SIOP PODC: recommendations for supportive care of children with cancer in a low-income setting. Pediatr Blood Cancer. 2013 Jun 1;60(6):899-904.

Gautam

Adhikari

. Palliative care services for cancer patients in Nepal, a lower-middle-income country. Palliat Care Soc Pract. 2021 Jan 1;15:26323524211021105.

Caruso Brown

Howard

Baker

Ribeiro

Lam

. Reported availability and gaps of pediatric palliative care in low- and middle-income countries: a systematic review of published data. J Palliat Med. 2014 Dec;17(12):1369-1383.

Hannon

Zimmermann

Knaul

Powell

Mwangi-Powell

Rodin

. Provision of palliative care in low- and middle-income countries: overcoming obstacles for effective treatment delivery. J Clin Oncol. 2016 Jan 1;34(1):62-68.

De Gagne

Koppel

Kim

Park

Rushton

. Pedagogical foundations of cybercivility in health professions education: a scoping review. BMC Med Educ [Internet]. 2021 [cited 2021 Mar 10];21. Available from: https://journals.scholarsportal.info/details/14726920/v21inone/nfp_pfocihpeasr.xml

10.

Sasaki

Bouesseau

Marston

Mori

. A scoping review of palliative care for children in low- and middle-income countries. BMC Palliat Care. 2017 Nov 25;16(1):60.

11.

Khan

Ahmad

Iqbal

Kamal

. Physicians knowledge and attitude of opioid availability, accessibility and use in pain management in Bangladesh. Bangladesh Med Res Counc Bull. 2014;40(1):18-24.

12.

Pandey

Gaire

Dhakal

Jaishwal

Kharel

Vaidya

. Perception of palliative care among medical students in a teaching hospital. JNMA J Nepal Med Assoc. 2015;53(198):113-117.

13.

Frehywot

Vovides

Talib

, et al. E-learning in medical education in resource constrained low- and middle-income countries. Hum Resour Health. 2013 Feb 4;11(1):4.

14.

Salins

. Need for palliative care education in India: can online palliative care education bridge these needs? Indian J Palliat Care. 2020;26(1):1-3.

15.

Arora

Smith T

Snead

Zalud-Cerrato

, et al Project ECHO: an effective means of increasing palliative care capacity. Am J Manag Care. 2017;23(7 Spec No.):SP267.

16.

Lalloo

Osei-Twum

Rapoport

, et al. Pediatric project ECHO®: A virtual community of practice to improve palliative care knowledge and self-efficacy among interprofessional health care providers. J Palliat Med [Internet]. 2020 Dec 16 [cited 2021 Jan 13]. Available from: https://www.liebertpub.com/doi/10.1089/jpm.2020.0496.

17.

Zhou

Crawford

Serhal

Kurdyak

Sockalingam

. The impact of project ECHO on participant and patient outcomes: a systematic review. Acad Med. 2016;91(10):1439-1461.

18.

Doherty

Rayala

Evans

Rowe

Rapelli

Palat

. Using virtual learning to build pediatric palliative care capacity in South Asia: Experiences of implementing a teleteaching and mentorship program (Project ECHO). JCO Glob Oncol [Internet]. 2021 Feb 8 [cited 2021 Feb 10]. Available from: https://ascopubs.org/doi/full/10.1200/GO.20.00481.

19.

McBain

Sousa

Rose

, et al. Impact of project ECHO models of medical tele-education: a systematic review. J Gen Intern Med. 2019 Dec;34(12):2842-2857.

20.

Doherty

Modanloo

Evans

, et al. Exploring health Professionals’ experiences with a virtual learning and mentoring program (project ECHO) on pediatric palliative care in south Asia. Glob Pediatr Health. 2021 Jan 1;8:2333794X211043061.

21.

Usher

Payne

Real

Carey

. Project ECHO: enhancing palliative care for primary care occupational therapists and physiotherapists in Ireland. Health Soc Care Community. 2022;30(3):1143-1153.

22.

Chen

Reeves

. Twelve tips for conducting educational design research in medical education. Med Teach. 2020 Sep 1;42(9):980-986.

23.

Peters

Adam

Alonge

Agyepong

Tran

. Implementation research: what it is and how to do it. Br Med J. 2013 Nov 20;347:f6753.

24.

Doherty

Okhuysen-Cawley

Chambers

. Children’s palliative care across a range of conditions, settings and resources. In: Downing

, ed. Children’s palliative care: an international case-based manual [internet]. Springer International Publishing; 2020 [cited 2020 May 13]:25-37. Available from: https://doi.org/10.1007/978-3-030-27375-0_3

25.

Doherty

Lynch-Godrei

Azad

, et al. Using virtual learning to develop palliative care skills among humanitarian health workers in the rohingya refugee response in Bangladesh. J Med Educ Curric Dev. 2022;9:23821205221096099.

26.

Struminger

Arora

Zalud-Cerrato

Lowrance

Ellerbrock

. Building virtual communities of practice for health. Lancet. 2017 Aug 12;390(10095):632–634.

27.

Vallath

Rajagopal

Perera

Khan

Paudel

Tisocki

. Access to pain relief and essential opioids in the WHO South-East Asia region: challenges in implementing drug reforms. WHO South-East Asia J Public Health. 2018;7(2):67.

28.

Pei

. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Med Educ Online. 2019;24(1):1666538.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.02 MB

0.15 MB