Abstract
Background:
Overdoses and alcohol consumption rose during the pandemic. However, uptake of practices which reduce mortality (eg, medications for opioid use disorder, harm reduction practices) remains insufficient. Provider training and telementoring is needed to ensure sufficient capacity for treating substance use disorders (SUDs) with evidence-based practices. The Project ECHO (Extension for Community Healthcare Outcomes) model involves the use of web technologies to deliver didactic and case-based learning through a panel of experts to build such competency in a community of learners. Project ECHO was leveraged to implement a statewide telementoring center of addictions-focused ECHO programs, including programming in prescribing, harm reduction, recovery support services, collaborations with first responders, and systems-level challenges.
Methods:
Participants represented health and behavioral health disciplines practicing across the state of Texas in metropolitan and rural areas. Learners were administered: (1) an online registration form that inquired about basic demographics, (2) a post-session survey at the conclusion of each session capturing satisfaction and likelihood to implement, and (3) annual surveys measuring changes in knowledge and self-efficacy. Attendance and other learner data were stored and extracted from the partner relationship management database: iECHO.
Results:
Training programs were attended by 968 learners, with an average of 48 learners per session. Geographic reach included 47 Texas cities. Post-training survey results indicated high rates of learner satisfaction, with an average rating of 4.68 on a 5-point Likert-like scale. Annual surveys indicated improvements in provider knowledge and self-confidence across all programs.
Conclusions:
Early results indicate robust uptake, wide geographic reach, high learner satisfaction, and provider knowledge and confidence gains. This preliminary evidence supports the use of the ECHO model as a potential tool for scaling comprehensive SUD telementoring centers to meet workforce development needs over large geographic areas.
Highlights
Fatal drug overdoses and alcohol-related deaths continue to rise.
Additional training and education is needed for health and behavioral health providers to address substance-related mortality.
Project ECHO provides an innovative model for building capacity in large numbers of providers spread across a vast region to address substance use.
Preliminary results of a substance use training center leveraging Project ECHO demonstrates promise for reach, learner satisfaction, and improvements in knowledge and confidence.
Introduction
Overdoses rose during the COVID-19 pandemic, with 107 573 deaths in 2022, the highest number ever recorded. 1 Alcohol-related deaths and alcohol consumption similarly rose during the pandemic.2,3 Texas specifically realized a significant increase in overdoses, with over 5000 drug overdoses from November 2021 to November 2022 and more than 600 000 residents suspected of meeting criteria for an opioid use disorder. 4 The substance use challenge is compounded in Texas, with 60% of counties designated as medically underserved, a lack of Medicaid expansion to support substance use disorder (SUD) treatment funding, and a rate of buprenorphine prescribers twice as low as the national average.5,6
Despite the scope of the crisis, uptake of evidence-based practices (EBPs) which reduce morbidity and mortality, such as medications for opioid use disorder (MOUD) and harm reduction strategies, remains insufficient.7-9 Widescale provider training and telementoring (training and coaching via virtual technologies) is needed to ensure sufficient preparation for treating SUDs. However, traditional training, such as time-limited workshops devoid of ongoing coaching or implementation support, often fail to result in widescale uptake of EBPs for SUD. 10
In contrast to traditional training, Project ECHO (Extension for Community Healthcare Outcomes) involves frequent, but brief, telementoring, inclusive of didactic and case-based learning. Project ECHO was developed in 2003 to build capacity across New Mexico to treat hepatitis C; the program has since expanded internationally to build provider capacity to address a myriad of conditions.11,12 ECHO programs are facilitated by a “hub” of experts, in which learners are exposed to instruction in clinical practices and consultation on complex patient cases. The combination of didactic and case-based learning, inclusive of feedback and coaching, is intended to build provider competence over time to deliver specialty care for the targeted condition.
ECHO historically addressed physical health conditions; it has increasingly been utilized to address behavioral health conditions like SUD. 13 Addictions-focused ECHO programs have shown promise for affecting provider ability to manage SUDs. SUD ECHO efforts in New Mexico coincided with a substantial increase in buprenorphine-waivered providers, resulting in the highest number of buprenorphine-waivered providers per capita of any US state. 14 Similarly, Colorado leveraged ECHO to expand utilization of MOUD, and found high rates of provider satisfaction and improvements in providers’ knowledge and self-reported ability to prescribe SUD medications. 15 Addiction medicine-focused ECHOs have additionally been used internationally, resulting in high rates of learner satisfaction and retention, as well as significant increases in knowledge and self-confidence in managing SUD. 16
ECHO has further been utilized to address systems-level SUD challenges, including population health approaches, provider resilience, supply chain challenges, and advancing integrated care. 17 Systematic reviews of SUD ECHO programs indicate strong program accessibility and utility, improvements in provider knowledge and self-efficacy, and clinical behavior changes (eg, buprenorphine prescribing). 18 Osei-Twum et al caution, however, that most ECHO evaluations have focused on provider knowledge and self-efficacy rather than direct patient outcomes, as ECHO programs do not directly target patients. Within their scoping review, Osei-Twum et al identified only 15 ECHO studies, of nearly 600 articles, that evaluated patient-level outcomes for various chronic health conditions targeted by ECHO programs. Nonetheless, where patient-level outcomes were available, improvements included reductions in hospitalizations, reductions in opioid prescribing, decreased mortality, and other patient-level health improvements, varying by ECHO program type. 19
This article describes the development of a statewide center for SUD telementoring in Texas that furthers the ECHO model by creating a continuum of SUD ECHO programs, with supplemental supports, that were responsive to local community needs. The goal of the telementoring center was to establish a comprehensive statewide infrastructure to reach and train the interprofessional Texas workforce on evidence-based SUD practices. The development of the SUD telementoring center lays the groundwork for our long-term goal of improving patient access to, and the quality of, SUD services across Texas through the growth and expansion of ECHO programming. We hypothesized that a telementoring center with multi-SUD ECHO programs, targeting different disciplines, and focused on critical practices with low uptake (eg, harm reduction, MOUD prescribing) would reach a large number of providers throughout Texas, result in high learner satisfaction, and foster significant improvements in provider knowledge and confidence—a demonstrated precursor to clinical behavior change.20,21 We believe our model offers the potential for replication in other states with high SUD-related mortality and limited SUD services.
Methods
The Center for Substance Use Training and Telementoring (C-STAT) was launched in 2022 in an academic medical center within a state academic health science center in a major metropolitan area of Texas. C-STAT includes a suite of 6 stigma-free ECHO programs focused on substance use, with varying programmatic focuses, such as a harm reduction, MOUD, implementation and sustainability, public safety interventions, and others, as informed by community-identified needs. In addition to ECHO programming, learners had access to webinars, an annual conference, and technical assistance. A quantitative evaluation approach, leveraging provider self-report, was used to assess the potential of the center to reach large portions of Texas, be well received by learners, and affect SUD knowledge and confidence levels among participants. The below outlines the approach for selecting telementoring topical areas, the learner recruitment process, and measures used to assess learner impact.
Needs Assessment for Program Development
Three ECHO programs were established prior to launching the telementoring center, a program focused on medications for SUD (originally “TxMOUD ECHO,” currently “TxRx ECHO,” and referred to as TxRx ECHO hereafter), a recovery support services ECHO (“RSS ECHO”), and an ECHO focused on hospital-based interventions (Support Hospital Opioid Use Disorder Treatment or “SHOUT ECHO”). These programs were launched over a 3-year period to provide training and support to a statewide network of SUD providers contracted through the university via funding from the state health department, with the intent of bolstering treatment capacity. In 2021, in response to rising overdose rates associated with the pandemic, the university was provided supplemental funding from the Health and Human Services department to rapidly deploy a comprehensive training institute, to meet the workforce needs across Texas beyond the existing provider network. To identify community priorities and obtain buy-in for the new center, a multifaceted needs assessment was conducted, including: (1) a literature review of SUD-focused telementoring efforts, (2) a modified nominal group technique (NGT) with SUD subject matter experts, and (3) an open discussion and priority ranking of topics identified in the NGT exercise with key community partners. Findings from these activities informed the new telementoring programming.
Capitalizing on the success of the prior ECHO programs, the existing and new programs were combined to form a comprehensive suite of interdisciplinary addictions-focused ECHO programs, under one all-inclusive telementoring center. In addition to ECHO programming, learners could attend webinars and a hybrid in-person/virtual 2-day conference, as well as request supplemental technical assistance. C-STAT received support and technical assistance throughout the implementation process from the local ECHO “Super Hub” (an ECHO center endorsed by the ECHO Institute to support implementation).
Learner and Expert Recruitment
Learners were recruited through a website, email list serves, “word of mouth,” and exhibiting at community events. Learners were provided with no-cost continuing education credits as an incentive to participate. Health and behavioral health providers practicing in a range of clinical settings in metropolitan and rural areas across Texas were invited to participate to ensure diverse and interdisciplinary participation (Note: a proportion of learners were from the host organization’s statewide network of SUD providers and were contractually bound to participate in ECHO). Hub members included university faculty and national SUD experts from various disciplines (eg, medical providers, behavioral health professionals, peer support staff) with differing areas of expertise (eg, direct clinical care, harm reduction services, implementation strategies).
ECHO Session Format
ECHO programs occurred monthly via Zoom, for 1 hour during the lunch hour to avoid disrupting clinic workflows. Each session included a 30-minute didactic presentation (eg, buprenorphine prescribing, patient-centered communication, overdose prevention), followed by a 30-minute learner case presentation. Following the case presentation, members of the hub provided recommendations for the patient’s care. At initial sessions of new programs, learners were oriented to the ECHO model (eg, format, learner expectations); ongoing sessions followed the standard ECHO model—a didactic presentation followed by a case presentation.
Data and Analytical Approach
The data and analytic approach were designed to evaluate preliminary outcomes of the telementoring center, by assessing program reach, learner satisfaction, and changes in provider knowledge and self-confidence to employ SUD services. Learners were included in the evaluation if they attended one or more sessions. Programmatic data were collected from Zoom registration reports, attendance records in iECHO (the ECHO Institute’s customer relationship database), and post-session and annual surveys.
Zoom registration forms captured contact information, demographics, and reasons for joining. ECHO attendance was captured through the Zoom attendance feature. Zoom attendance records were reviewed, reconciled, and uploaded into iECHO. Instructors and staff were excluded. Using probabilistic matching, Zoom registration reports and iECHO attendance records were combined and reconciled to assess program reach and learner demographics.
Post-session surveys, sent to learners in all programs, were designed to evaluate satisfaction with the programming, as well as intent and confidence in application. Surveys were administered online via REDcap, an electronic data capture software, following each session. Surveys probed on various dimensions of programming, including unbiased material, presenter preparedness, and applicability of content. Responses were gathered using a 1 to 5 Likert-like scale, with 5 indicating very dissatisfied/strongly disagree/not confident at all. For analysis the scale was reverse coded, where 5 represented very satisfied/strongly agree/very confident. Key items included: (1) “How satisfied are you with the overall quality of this session,” (2) “The material presented will be useful to me in implementing [X] practices,” (3) “How confident are you that you could implement the evidence-based practice/s in today’s session,” and (4) “I would recommend this ECHO program to a colleague.” Mean scores for each item were calculated across all ECHO programs, providing insight into the effectiveness and impact of each session.
Annual retrospective pre/post surveys were designed to assess learners’ self-reported change in knowledge and confidence. For new programs, surveys were administered on an annual basis; for existing programs that had been operating 1 or more years, the surveys were conducted approximately 3 months after the telementoring center’s inception. Annual follow-up surveys included uniform items across programs, while incorporating program-specific questions to gauge knowledge and confidence. Indicators of learners’ knowledge and confidence stemmed from program learning objectives and utilized a 1 to 5 Likert-like scale, where higher scores indicated greater levels of knowledge or confidence. Mean scores were calculated across indicators. We analyzed shifts in pre/post levels of knowledge and confidence by determining the percent change from pre- to post-attendance.
Results
Needs Assessment Findings
Needs assessment activities (literature review, NGT, and community meetings) identified priority areas for new programming, including a harm reduction ECHO (“Harm Reduction ECHO”), a public safety professionals-focused ECHO (“Public Safety Solutions for SUD ECHO”), and a leadership-facing ECHO focused on systems-level and organizational SUD challenges (“Systems and Sustainability ECHO”). These were consolidated with existing ECHO programs to form the suite of SUD ECHO programming. Along with identified targets for additional ECHO programs, the needs assessment informed thematic threads to be interwoven across all offerings, including: health equity, social determinants of health, stigma, and patient-centered care. These themes were incorporated into didactics, case presentations, and discussion for all programs.
Attendance and Engagement
From March 1, 2022, through August 31, 2023, 968 unique learners participated in ECHO programming. An average of 48 learners participated in each session. Of participants who provided demographic information, the majority were women (55.9%) of Caucasian/White descent (57.7%). Other racial identities included African-American/Black (10.8%), with American Indian/Alaskan Native, Asian, and “other” categories representing 5% of learners. A quarter (25.7%) of learners reported Hispanic ethnicity. Learners’ professional backgrounds were diverse with the most common being behavioral health provider (21.7%), followed by peer specialist (19.2%), and physician (6.9%). Remaining providers included public safety professionals, advanced practice providers, nurses, and others. Table 1 provides an overview of learner demographics. Note, providing demographics was optional and some declined to disclose.
ECHO Participant Demographics March 2022 to August 2023.
Learners were not required to provide demographics, captured at the time of registration, to participate in the program, thus resulting in some missing data.
Abbreviation: ECHO, Extension for Community Healthcare Outcomes.
ECHO sessions demonstrated wide uptake across Texas, with learners located in 47 cities and all Texas public health regions (state-designated service delivery regions). Figure 1 displays the number of cities where ECHO learners were located, overlaid on public health regions. Sites were clustered around major metropolitan areas, but were notably disbursed into rural regions, including the Texas Panhandle, coastal areas, and border region.
Texas cities containing ECHO learners, March 2022 to August 2023.
Post-Session Surveys
Across all programs, the average post-session survey response rate was approximately 23%. Post-session survey results indicated high learner satisfaction (mean [M] = 4.68, standard deviation [SD] = 0.62, N = 1984) and a high likelihood of referring colleagues to the program (M = 4.75, SD = 0.50, N = 1950). Learners also reported high confidence in implementing practices from the sessions into clinical practice (M = 4.44, SD = 0.72, N = 1995) and found strong utility in materials for supporting implementation (M = 4.69, SD = 0.58, N = 1946). Note: the total sample size for post-session surveys is higher than the total number of ECHO learners as learners were asked to complete the survey for all sessions across all programs they attended.
Annual Retrospective Pre/Post Surveys
All program attendees were asked to complete a more in-depth annual survey, with response rates varying by program: RSS ECHO at 25%, Harm Reduction ECHO at 10%, SHOUT ECHO 21%, Systems and Sustainability ECHO at 25%, and TxRx ECHO at 12%. The average response rate across all biannual surveys was 18.6%. Learners reported improvements in knowledge of SUD principles and confidence in executing approaches for SUD treatment. Individual knowledge and confidence items were collapsed into one mean score per program and domain, and percent change was calculated pre- to post-ECHO. Program-specific knowledge increased across all programs, with Recovery Support Services, Systems and Sustainability, and SHOUT ECHOs showing increases in knowledge of 60%, 52%, and 57%, respectively. This was followed by Harm Reduction ECHO (28%) and TxRx ECHO (22%). Confidence in applying these approaches also rose, with Systems and Sustainability ECHO experiencing the highest increase (47%), followed by SHOUT ECHO (44%), RSS ECHO (36%), Harm Reduction ECHO (27%), and TxRx ECHO (9%). For prescriber-specific feedback, SHOUT and TxRx ECHOs demonstrated increases in confidence of 32% and 25%, respectively. Percent changes across knowledge and confidence domains are presented in Tables 2 and 3. Individual knowledge and confidence items are detailed in the Supplemental Appendix.
Percent Change in Perceived Knowledge Before and After ECHO.
Abbreviations: ECHO, Extension for Community Healthcare Outcomes; SD, standard deviation; TxRx, Tx=Texas, Rx=prescriptions.
Percent Change in Perceived Confidence Before and After ECHO.
Abbreviations: ECHO, Extension for Community Healthcare Outcomes; SD, standard deviation; TxRx, Tx=Texas, Rx=prescriptions.
Conclusion
With elevated overdose rates and increasing alcohol-related deaths, there is continued need for enhanced workforce development efforts to prepare clinicians to effectively address SUD. However, traditional training is often inconsistent with best practices for effective implementation of SUD services due to a lack of coaching, mentoring, or other forms of technical assistance and support, and a limited focus on case-based, applied learning. The ECHO model provides an accessible means of ongoing implementation support and telementoring, as well as a focus on application and skill building through case presentations and consultation.
Preliminary outcomes of the ECHO-centered SUD telementoring center indicated far reach of the programming statewide, with 968 learners participating from 47 Texas cities. While the volume of learners was focused in metropolitan areas, more than a fifth (21%) of identified cities in which ECHO learners were participating are designated rural areas. Thus, ECHO programming proved accessible across the state, including rural areas, without the need for proximity to the academic medical center host; this is consistent with the broader literature indicating ECHO is a useful model for reaching those who might otherwise have to travel long distances for high-quality training. 22
In addition to broad reach and participation, learners were highly satisfied with the programming, with an average satisfaction rating of 4.69 out of 5 (very satisfied) across all programs. This may be due to a number of factors, including: accessibility (available online via Zoom), incentives to participate (continuing education credits), relevance to one’s day-to-day clinical work (as seen in post-session survey responses), or reduced professional isolation. 23
The preliminary evaluation of learner impact revealed improvements in knowledge of key practices and increased confidence to implement those practices, which has been demonstrated in the literature to precede later changes in clinical behavior and implementation.20,21(p5) Learners reported knowledge gains in all programs, but the greatest knowledge gains were realized in the RSS ECHO, in which a 60.48% increase in knowledge of key learning objectives after ECHO was reported. Learners additionally demonstrated increased confidence in applying key concepts across all programs, with the greatest improvements seen in the Systems and Sustainability ECHO, with a 46.8% increase in confidence, and SHOUT ECHO with a 43.78% increase in confidence. Greater knowledge gains in RSS ECHO and confidence gains in SHOUT ECHO programs may be explained by training dosage, as these were in operation prior to the development of the telementoring center, thus learners have had more time to participate, as compared to other programming. As seen across the ECHO literature, didactic and case-based learning, which teaches key concepts and demonstrates the application of that knowledge to a real-world setting, may support knowledge transfer and confidence to apply that newly acquired knowledge. 24
Findings from the preliminary evaluation suggest the programming model used by C-STAT could be leveraged for community-facing SUD telementoring programs in other states, informed by local community needs. While uptake of critical evidence-based SUD practices, such as MOUD or harm reduction, remain insufficient nationally, this model provides a framework for building training and mentoring programs in SUD that has demonstrated strong reach, high learner satisfaction, and provider knowledge and confidence gains. This expands current SUD ECHO literature, which largely focuses on singular EBP-specific ECHO programs (eg, to increase MOUD utilization) rather than a broader suite of programming intended to reach a large, interdisciplinary audience. The C-STAT model gave practitioners exposure to a broad range of SUD topical areas on a recurring basis (6 offerings monthly) including prescribing practices, harm reduction practices, setting-specific interventions (eg, interventions in hospital settings), strategies for cross-sector collaborations, and a means for addressing macrolevel challenges within their organization or practice region, supplemented by on-demand technical assistance and other forms of telementoring outside of ECHO (eg, conferences and webinars).
Limitations of this study include low participation of physicians (7% of participants), particularly critical to advancing medications for SUD associated with reducing mortality. 25 Previous evaluation of low physician participation in SUD training has included stigma, reimbursement concerns, and a lack of support from colleagues. 26 To encourage physician participation, sessions were offered over a typical clinic lunch hour (noon) and free continuing medical education credits were offered. More physician-specific recruitment efforts will be leveraged at the training center going forward, such as recruitment via physician professional guilds.
Other limitations include the retrospective approach taken for annual surveys to measure changes in knowledge and confidence which is subject to recall bias. The retrospective pre/post design was employed due to the open enrollment nature of the programming, which poses challenges to capturing a point-in-time baseline measure. This method was also selected to avoid a barrier or disincentive for learners at the time of program registration (ie, being required to complete a lengthy survey to participate in telementoring). Further limitations to this approach include reliance on provider self-report and not actual patient outcomes. The collection of patient outcomes to evaluate ECHO programs has proved challenging due to participation of diverse organizations with different data structures, making the attainment of patient-level data costly and minimally feasible (eg, the difficulty of establishing data-sharing agreements with a large number of organizations). Future research efforts related to evaluating centers like C-STAT should, resources permitting, include an examination of patient-level outcomes and impacts (eg, patient receipt of MOUD, reduced substance use, retention in treatment, etc) which may require a smaller demonstration project within a larger ECHO system, in which health systems amenable to sharing patient-level data opt in and standardized reporting measures are collected.
Further limitations included a low sample size for annual surveys when compared to the total learner sample. This may be due to provider time constraints (eg, due to high patient volume and limited time for extra activities) or insufficient incentives. Future studies could include higher monetary incentives and time within live ECHO sessions to complete.
Despite study limitations, preliminary outcomes from this evaluation demonstrate a robust ECHO-based SUD telementoring center which leverages the multifaceted approach taken by C-STAT (ie, community input into program planning, a suite of ECHO programs as opposed to a singular SUD ECHO, and additional supports outside of ECHO), can be leveraged to reach a large geographic area inclusive of metropolitan and rural jurisdictions, while maintaining high rates of learner satisfaction, and resulting in improvements in critical SUD treatment knowledge and the confidence to carry out those evidence-based SUD practices. Thus, ECHO and telementoring more broadly may serve as a useful tool to address widescale SUD workforce development in other states, to address the increasing rates of substance use and substance-related deaths, and the enduring workforce knowledge and practice gaps in the area of substance use.
Supplemental Material
sj-docx-1-saj-10.1177_29767342241252296 – Supplemental material for Leveraging Project ECHO to Implement a Suite of Substance Use Learning Communities for Statewide Impact
Supplemental material, sj-docx-1-saj-10.1177_29767342241252296 for Leveraging Project ECHO to Implement a Suite of Substance Use Learning Communities for Statewide Impact by Adrienne C. Lindsey, Carma Deem-Bolton, Erin Finley, Jennifer Sharpe Potter, Holly Lanham and Sanjuana Fleming in Substance Use & Addiction Journal
Footnotes
Author Contributions
Dr Lindsey served as Center Director during this timeframe, contributed to the implementation and operations of the training program, and wrote and edited the manuscript. Drs Finley, Sharpe-Potter, and Lanham contributed to the design of the telementoring program, the theoretical framework of the manuscript, and advised on the analysis and broader implications for the field in the conclusions. Dr Deem-Bolton oversaw the evaluation of the project and wrote the methods and results sections. Ms Fleming conducted data collection for the project and contributed to the analysis. All authors contributed to writing and editing the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Center for Substance Use Training and Telementoring is supported by the Texas Health and Human Services Commission through federal funding from the Substance Abuse and Mental Health Services Administration grant award number HHS001196700002.
Compliance,Ethical Standards,and Ethical Approval
Institutional Review Board approval was not required.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
