The perceived impact of project ECHO networks in Northern Ireland for health and social care providers,patients,and the health system: A qualitative analysis

Abstract

This study assesses the perceived impact and benefits of Project ECHO (Extension for Community Healthcare Outcomes), a tele-mentoring intervention for health and social care providers, patients and the health system in Northern Ireland. Having access to a specialist, a space to share experiences, and being able to disseminate up-to-date best practice were all cited as improving provider knowledge as well as improving quality of care for patients. Healthcare providers reported being more confident in managing patients and that relationships had been improved between different levels of the health system. ECHO was described as improving access to education and training by removing geographic and time barriers. This is one of the first studies to qualitatively analyse impact across a number of different clinical and social care ECHO networks. The results strongly indicate the perceived benefit of ECHO in improving provider, patient and health system outcomes such as increased healthcare provider knowledge and confidence to manage patients at primary levels of the health system. This has implications for future service design, particularly within the context of COVID-19 in which virtual and online training is necessitated by social distancing requirements.

Introduction

Project ECHO is an internationally recognised tele-mentoring intervention that has been implemented in over 40 countries. ¹ ECHO connects specialist healthcare providers (hubs) with providers in primary and community practice (spokes), often in rural and isolated areas.^2,3 Using video-conferencing technology, providers gain access to evidence-based best practice guidance from specialists. They have the opportunity to participate in case-based learning, share their own experiences, ask questions, and make connections with other healthcare providers in their areas of expertise. The ECHO model was developed at the University of New Mexico, USA, and is underpinned by the principles of access, de-monopolisation and de-centralisation of knowledge, building connections across the health workforce and building capacity in primary care. ⁴

Previous evaluations of ECHO consistently demonstrate positive impacts of the programme on healthcare provider outcomes. Two systematic reviews have been conducted that assess the outcomes of studies evaluating the impact of Project ECHO across different health conditions and networks.^5,6 Both reviews indicated positive results regarding the impact of ECHO on increased participant knowledge, competence, confidence, and satisfaction. Positive impacts have also been demonstrated for patient outcomes such as improved care and reduced waiting times, as has been seen in networks focusing on chronic liver disease, ⁷ and declines in opioid prescribing in managing pain in patients. ⁸

To date, however, few qualitative studies have been conducted to try and identify crosscutting themes, factors and impact across a large range of different clinical and social care networks, and including Network participants (spokes), hub members, leads, facilitators and educators. The inclusion of these diverse participants and hub members, and that this study was conducted in a ‘real-world’ setting in which ECHO was embedded into health service delivery, ensures this study adds important value to the existing literature by addressing important research gaps and questions.

In 2018, Hospice UK was commissioned by the Health and Social Care Board in Northern Ireland to deliver services for the provision and facilitation of 30 Project ECHO knowledge networks. Each ECHO Network supports providers in specific areas of expertise, for example neurology, cardiology, or prison health care. Networks were both clinical and social care related, and are led by experienced practitioners within the area of specialism, who support other network participants to set a curriculum of study and to facilitate different presentations within the Network. Network leads are often based in ‘hubs’, or regional centres where a multidisciplinary team of subject matter experts is located. The aim of this study was to qualitatively examine the perceived impact and benefit of the networks in supporting improved provider, patient and health system outcomes as described by their participants.

Methods

The study followed a descriptive qualitative approach. ⁹ Purposive sampling was used to identify network leads, facilitators (who support the administration and running of ECHO sessions) and educators (individuals who are brought in to provide training, support and expertise on a particular subject matter) across different health and social care contexts, and both frequently and infrequently attending participants to capture a wide range of perspectives on Project ECHO networks. Attendance records from each network were used to identify low and high attenders, with eligible participants contacted by email and recruited until saturation was reached. Questions were formulated with reference to the Consolidated Framework for Implementation Research ¹⁰ to ensure they were both systematic and designed with a view to supporting a rapid and pragmatic service evaluation.

Semi-structured telephone interviews were conducted with networks leads and participants between January and March 2020. Interviews lasted between 15 and 45 min. Telephone interviews were completed by the evaluator, digitally recorded and transcribed verbatim. Data were analysed thematically according to Braun and Clarke (2006) (Familiarisation; coding; generation of initial themes; reviewing; refining and defining). ¹¹ A flexible approach incorporating both inductive and deductive approaches was used, ¹² with deductive categories and codes identified from the ECHO 101 Evaluation Guide (Processes and Outcomes). ¹³ Sub analysis was conducted on how additional variables impacting on perceived benefit of the ECHO model. These variables were inductively identified from the thematic analysis. The study is reported according to the Consolidated Criteria for Reporting Qualitative Studies. ¹⁴

Results

Twenty-seven participants took part in the study, eight of whom were network leads, four were facilitators/educators and 15 were network participants. A total of 15 different networks were represented in the sample. Participants indicated that the perceived impact of ECHO was interconnected across outcomes for providers, patients and the health system. For example, increased provider knowledge was described as improving treatment for patients, and enabled more efficient referral and management of patients. A number of variables were identified relating to the quality of both participation and outcomes generated from the ECHO networks, with the importance of leadership and support from senior managers being consistently cited. The ECHO methodology was described as useful in reducing barriers to participation in training and communities of practice, while the support of the ECHO team in providing both technical and administrative support to each network was equally recognised.

Improved knowledge, confidence and connection among healthcare providers

Across all networks, participants described improved knowledge through engaging within the network. Improved knowledge was reported relating to a range of areas including clinical knowledge, symptom recognition, patient management, awareness of up-to-date best practice guidelines and new treatments, a better understanding of referral processes, an improved understanding of the different levels and structures within the health system, and an awareness of new research. Participants described benefiting from both access to specialist knowledge, as well as from sharing experiences and peer-learning. Case studies and the opportunity to discuss specific challenges were frequently cited as useful exercises conducted within the sessions.

“I thought the case presentations were interesting and you learnt as much from everyone joining in and giving their views and the conversation after and that is just as much where the learning is - the peer discussions, after the presentations, I found that useful and helpful and it got me thinking.” (002)

“I gained a lot from it, probably more from the experts, but also from hearing other people’s problems and how they solved those problems so I got a lot out of it” (014)

“It’s a great way of learning and quite up-to-date, because sometimes the consultants are sharing very fresh research findings or something that has happened in their work environment so you are quite up-to-date… The consultants at that level are very good at giving ‘clinical pearls’ there is a lot you will get from listening to an experienced practitioner that you won’t get in a textbook or a journal you know” (016)

“Speaking with other members in different areas dealing with the same situation and coming up with solutions whether it be symptom management or breathlessness, bowel obstruction or whatever you know. It was really homing in on patients similar to mine and maybe things that I hadn’t thought about.” (018)

Nearly all participants felt like the information they gained through the sessions would be useful and they had/would apply it within their own practices. The sessions were described as having helped to improve confidence in managing patients and in improving providers’ self-efficacy. One participant, however, raised a concern about the wider impact of the sessions beyond the existing network, the application of knowledge, and a lack of further dissemination.

“I have been able to impart (a lot of information) over to my patients. Information that we didn’t have before about the hospital services so we can say actually “this is what is going to happen, this is the timeline” and then of course e-referral has come in the middle of this as well, so there have been so many changes its actually difficult to pick up on them all.” (023)

“I wouldn’t have felt either confident or competent enough to provide that service if I hadn’t have gone those 3 years with the continual knowledge building and that gradual increase of knowledge that I built through ECHO” (019)

“I could see that the participants were interested and they were listening. I didn’t get the feeling that they were going to do anything with the information. I didn’t get the feeling that they were going to disseminate that information out to all of their colleagues, and perhaps nurses and other people like that.” (009)

Beyond having access to specialist knowledge and the opportunity to benefit from peer-learning, participants across the networks described the benefits of being part of a community of practice. Particularly for participants working in isolated and remote areas, this community was described as being important in terms of improved practice, improved professional satisfaction, and learning across disciplines and levels of the health system, and developing a feeling of support and connection between colleagues.

“The people we invited to join Project Echo where the most remote District Nurses teams, with regards to the Peninsula, Donaghadee and Mourne Areas, people at the edge of the trust that still need support” (013)

“Whenever, I interact with any other Children’s Hospice or children’s public health service, it’s that feeling that we are all going through the same thing sometimes in small island communities, we think do other people have these issues and whenever you link into other people it’s reassuring to know it’s not just us that has these issues crop up and everybody is going through the same process and sharing the case studies is really useful to see how other people handled things or to put your own team ideas forward and you experience its really nice to have that community.” (015)

Improved quality of care and efficiency of referral for patients

A number of specific and diverse examples demonstrated improved patient outcomes as a direct result of ECHO participation. Members of the Ophthalmology networks (a more mature network in terms of their duration) described improved processes such as connecting ophthalmologists and GPs to sign off on a patient receiving a new treatment in order to preventing sight loss. One participant described being able to more efficiently access a brain scan for a patient with migraines and other potential ‘red flag’ symptoms. Through efficiently accessing referral, the patient was seen quickly and avoided a long and potentially worrying waiting period. Participants working in end-of-life care described learning and applying knowledge on best practice regarding managing side effects of pain treatments, as well as how to adapt the physical space around the patient to improve experience and comfort. Example excerpts of the interviews relating to improved patient outcomes are displayed in Figure 1.

Figure 1.

Patient outcomes demonstrated through the ECHO networks.

Reduced referral and progression towards strategic health system outcomes

Perceived benefits to the health system were attributed to the learning and connections developed through the ECHO networks. Improved confidence in providers’ skills, knowledge and management of patients, resulted in participants describing being more confident and efficient in their use of referrals. At times this was described at helping to reduce unnecessary referrals and ensure patients were treated in primary care, and in other examples this was described as having a better understanding of referral pathways thus allowing more efficient referral and reducing waiting times for patients. One participant however noted that given the scale of challenges regarding waiting lists, that in order to have a significant impact on reducing waiting lists ECHO would need to be dramatically scaled.

“It made practitioners more relaxed in their own clinical primary care setting and probably raised the bar for referrals. I imagine a lot of people feel that way as well more responsibility and monitoring situations rather than straight to referral.” (016)

“One scenario in particular (was getting) direct access through a specific pathway, which meant that I was able to get things arranged, reassure a patient and he is now no longer on a waiting list” (005)

“I don’t think it will help us to that level (reducing waiting lists), it’s more helping us deal with cases we are seeing on a daily basis. You could only reduce waiting lists when you are speaking to over 500 GPs but that’s not going to happen” (007)

Participants also reported ECHO as providing a space to discuss, identify, and progress towards different strategic objectives both within their own areas and across the system. Participants described significant and embedded changes in policy and practice that emerged from the networks, including on development of new clinical and/or practice guidelines.

“Our Minister of Justice at the time had asked for a review of vulnerable prisoners and so that review was on-going and documentation was being around the pathway for the vulnerable people. Our ECHO fed into that and the educators would bring “Best Practice and Guidelines” themselves the manifestation of that was there was learning as a result of difficult conversations that happened within the ECHO. Then we produced the self-harm framework and that was the time we had collectively looked at what where we doing and what can we do?” (011)

Factors affecting impact of ECHO

Three factors were identified by participants as important in influencing the impact of ECHO for providers, patients and the health system: the importance of the on-boarding, network initiation, and support from senior management; ECHO as reducing barriers to participation in training and education and thus allowing wider access to the network and extending the reach of positive outcomes; and the importance of leadership within networks.

ECHO as a model capable of reducing barriers to participation and improving quality of learning

As a methodology, ECHO was described as highly effective in reducing barriers to participating in training, and as a vehicle to establishing peer-to-peer networks and communities of practice. ECHO as an approach allowed the extension of the reach of the programme by removing geographic barriers and reducing time needed away from frontline work to attend training. Participants, particularly community workers in remote and rural areas, described ECHO not only as a tool for increasing access to education, but also as a way of connecting with colleagues and reducing isolation. For large teams who wanted to attend sessions, but couldn’t all leave their site for traditional training, ECHO was particularly appreciated.

“I think the obvious advantage is that people can do it onsite and that they don’t have to travel. Particularly if you’re doing it with teams it can be hard. We find it was becoming a bit of a challenge getting the Team to a physical location but with this you could maybe do it more frequently and they didn’t have to travel off site” (006)

“They could be travelling for over an hour to get to an education session and that is a lot of their day” (012)

“I live in a very rural area, so the other thing was the Networking with other members of teams in various other areas that may not be so accessible” (018).

The presence of the ECHO team to help facilitate and moderate sessions, to support with technical set-up and assistance, and to provide pre-session facilitation training and support, was described as highly important in ensuring sessions ran smoothly. The importance of IT support was highly stressed by a number of participants.

“The support from ECHO is very good and we couldn’t do it without that support along with the IT Support” (012).

“With ECHO you have the whole thing up and running and easy to engage with. Whereas, doing it yourself is a lot more challenging” (004).

“Whenever we have other face to face the technology is always down to us, there is always things going wrong, like videos don’t play on the computer or won’t link to the screen there is always that worry, so the IT team certainly takes that worry away” (013)

While such factors of IT support were cited by networks that reported greater impact and outcomes, participants also highlighted suggestions for how to further reduce access barriers. These included ensuring convenient times for meetings, providing further IT support for spokes and participants, addressing IT challenges through HSC systems, and better utilising online platforms in which resources were shared after sessions.

The importance of on boarding, initiation and support from senior management

Providing space for discussions, information sharing, and for building connections was described as essential in getting networks off the ground, and for establishing good practice that would underpin the following sessions. Given how busy healthcare providers can be, this space was described as vital in allowing networks to develop. Support and encouragement from senior managers and from supervisors was described as an important variable in encouraging participation, initiation and sustaining the different networks. Participants described not fully understanding what ECHO was, having concerns about using the technology and not having the time to nurture relationships as barriers to the performance and effectiveness of the network.

“Yes there was a lot of building “will” because like most service most people are acutely running around responding to crisis and have very little reflection or time to think if there a better way to do things. There wasn’t a resistance it was more just not even considering that we could do something like this” (011).

“We are lucky to be afforded it, again I was quite lucky because my bosses (I don’t own my own practice) gave me the time because they believed in what was being achieved” (023)

“I have to say one of my colleagues in the hub was sceptical about their support. I was ok about it as it would be good to have them as hopefully they will spread the word, but the fact is they didn’t attend or spread the word. It is important for them to really understand what Project ECHO is all about” (024)

The importance of leadership within networks

Leadership in each network was consistently cited as crucially important for ensuring networks were effective and efficient. Leadership that provided organisation and direction were important, in addition to instilling a sense of ownership and trust across participants in the network. Leaders who were able to utilise their own networks to invite key educators to the ECHO sessions, was also considered to be an important attribute. Participants in positions of leadership also noted the challenges on their time and the difficulties of completing the role when coupled with other clinical responsibilities.

“My own background was helpful. I had the contacts in a sensitive subject for some of the sessions subjects that where coming up. If I hadn’t had that background, I would have found it difficult” (003)

“You do have a responsibility and you need to keep tabs on what’s going on month to month and if the case studies have come in and so on, there needs to be a degree of commitment and organisation” (024)

“I really enjoyed it. It took more time than I anticipated and that was fine as I was in a job where that was the service and funded to do so, where if I had been in a busy, busy clinical role would that have been too much to chase those up.” (026)

Discussion

Participants described benefits of ECHO to their own knowledge, practice and self-efficacy; to patients; and to the health system. These benefits were often underpinned by variables affecting the creation of networks and participation within them, as well as the benefits of ECHO methodology in reducing barriers and providing organisational support. Impact may arise from the information a participant learns during a session, but it may also come from indirect moments and interactions, fostered by ECHO but outside of the formal space of the ECHO sessions. ECHO provides a space from which communities of practice and relationships develop and connections are made.

Benefits of ECHO also appear to be greatest in networks that have operated for extended period of time, for example the Ophthalmology Network and Prisons Healthcare Network, in which participants described the impact of ECHO in facilitating long-term goals around service transformation. This has important implications for long-term support and funding of Networks, as well as the need to ensure Networks identify specific and measurable objectives.

The IT and Administrative support built into ECHO was also essential in delivering high quality, structured education in a dynamic and facilitated manner. In a post-COVID-19 online world, many of the negative aspects of virtual learning, such as IT problems, lack of facilitation and organisation may be overcome using the ECHO approach and/or principles. Beyond providing infrastructural support, ECHO models the principles of co-creation and production, builds relationships and connects levels of the health system. This participatory, non-hierarchical nature of ECHO is also what distinguishes ECHO from many of the other training and webinar options that exist online. All of these are necessary elements in achieving long-term health service transformation and reform. ¹⁶ While all online learning models and webinars may reduce geographic barriers, the particular elements built into ECHO help to ensure the quality of the sessions are of a high standard, which in turn enhances the longer-term engagement of network participants.

Given the impact of COVID-19 in necessitating and normalising online learning, ¹⁵ ECHO provides an important model for how to ensure impact in a saturated online space in which many different webinars, online training tools, and approaches are available. This study demonstrated the importance of having a supportive infrastructure to support the delivery of online learning and also to foster the development of an online community. Network leads, supported with appropriate facilitation training, were essential elements of networks that functioned well and with impact.

The vast majority of participants in this study were positive about their experiences of ECHO, however a number of areas are require consideration when strengthening this project in Northern Ireland, and for teams looking to implement ECHO in other contexts. Consideration is required regarding securing support from senior management for the programme, a theme that appeared as an important variable regarding participation within different networks. Equally important is ensuring knowledge shared widely by participants, and that Networks have clear aims and objectives that relate to wider strategic objectives regarding service transformation.

Conclusion

Project ECHO supports the learning, self-efficacy and confidence of healthcare providers participating in both clinical and social care networks. Specific elements of the programme such as IT and Administrative support, and facilitation training for Network leads, are important in delivering high quality sessions, and suggest the utility of this approach in the context of COVID-19 in which a proliferation of online and virtual learning options now exist. This study also suggests important benefits to both patients and the health system, further indicating the wider service transformation potential of ECHO.

Limitations

In order to ensure the confidentiality of participants within the study, it is not possible to attribute quotes to specific networks, thus potentially limiting the reading of the study. Quantitative data would allow a wider analysis and triangulation with data presented in this study to improve confidence in results presented within this paper.