Sage Journals: Discover world-class research

Abstract

Introduction

Inadequate knowledge and training of healthcare providers are obstacles to effective chronic pain management. ECHO (extension for community healthcare outcomes) uses case-based learning and videoconferencing to connect specialists with providers in underserved areas. ECHO aims to increase capacity in managing complex cases in areas with poor access to specialists.

Methods

A pre-post study was conducted to evaluate the impact of ECHO on healthcare providers’ self-efficacy, knowledge and satisfaction. Type of profession, presenting a case, and number of sessions attended were examined as potential factors that may influence the outcomes

Results

From June 2014 to March 2017, 296 primary care healthcare providers attended ECHO, 264 were eligible for the study, 170 (64%) completed the pre-ECHO questionnaire and 119 completed post-ECHO questionnaires. Participants were physicians (34%), nurse practitioners (21%), pharmacists (13%) and allied health professionals (32%). Participants attended a mean of 15 ± 9.19 sessions. There was a significant increase in self-efficacy (p < 0.0001) and knowledge (p < 0.0001). Self-efficacy improvement was significantly higher among physicians, physician assistants and nurse practitioners than the non-prescribers group (p = 0.03). On average, 96% of participants were satisfied with ECHO. Satisfaction was higher among those who presented cases and attended more sessions.

Discussion

This study shows that ECHO improved providers’ self-efficacy and knowledge. We evaluated outcomes from a multidisciplinary group of providers practicing in Ontario. This diversity supports the generalisability of our findings. Therefore, we suggest that this project may be used as a template for creating other educational programs on other medical topics.

Keywords

Pain education tele-education tele-mentoring rural underserved remote chronic pain management project echo continuing medical education interprofessional medical education quantitative research methods community based education primary care education problem-based learning

Introduction

Chronic pain (CP) affects 20% to 30% of the adult population.^1–3 Worldwide, CP is among the top chronic conditions that contribute to years lived with distress and disability.⁴ Compared with people who live close to academic hospitals in urban areas, CP patients who live in rural areas often suffer from higher pain levels and more severe functional impairments, which are most likely mediated by depression and low quality of life.^5–7 In Canada, healthcare providers (HCPs) manage around two-thirds of CP patients in primary care settings, with little or no training in managing this condition.^8,9

Of particular concern in pain management is the dramatic increase in the rates of opioid prescriptions and the parallel rise of prescription opioid overdose, abuse and diversion.^7,10,11 Most opioid prescriptions are written by primary care clinicians who have received minimal training in safe opioid prescribing.^12,13

Project ECHO (extension for community healthcare outcomes) is a collaborative model of tele-education and care management that empowers participating clinicians to provide expert level care in their own communities. The first ECHO for CP started in New Mexico (USA) and it has consistently demonstrated improvements in self-reported knowledge, skills and advanced practice.^14,15 The replication of ECHO pain in the US Veterans Health Administration showed that participation in ECHO was associated with increased utilisation of physical medicine services and initiation of nonopioid medications among patients with chronic pain.¹⁶ The ECHO model™ does not ‘deliver’ specialist care directly to patients; instead, it increases access to specialist skills by providing primary care clinicians with the knowledge and support they need to manage patients living in rural or otherwise underserved areas. Evidence for the effectiveness of the ECHO model has been demonstrated in conditions such as hepatitis C, chronic pain and addiction.^17–19

In response to the difficulties faced by HCPs in managing patients with CP plus the opioid crisis in Ontario, the Ministry of Health and Long Term Care (MOHLTC) funded the first replication of the ECHO model in Canada: ECHO Ontario Chronic Pain/Opioid Stewardship; as of April 2017, this programme has received permanent funding from the MOHLTC.²⁰

The objectives of this study are to: (a) describe the impact of ECHO on clinicians’ self-efficacy and knowledge related to CP management and opioid prescribing; (b) evaluate acceptability and the impact of participation in ECHO on participant’s practices; and (c) assess if the results are modified by participant’s profession, number of sessions attended or presenting a case in ECHO.

Methods

Intervention

ECHO Ontario Chronic Pain/Opioid Stewardship includes an interprofessional group of CP experts, the ‘hub’. The hub connects with multiple primary healthcare providers, the ‘spokes’, in Ontario using videoconference once a week. The hub team includes physicians (pain medicine, addiction, family medicine, neurology, physiatry and psychiatry) as well as a psychologist, nurse, social worker, physical therapist, occupational therapist, pharmacist, chiropractor and medical librarian, along with a telemedicine technician. ECHO runs weekly 2-hour sessions that comprise a 20–30 min didactic on a relevant topic given by a Hub member, or invited speaker as well as discussion of actual cases presented by a participating spokes. The de-identified patient case presentation follows a standard format. The spokes can attend as many sessions as they want, and they inform the coordinator when they will exit the programme.

Between June 2014 and March 2017, ECHO delivered a total of 124 didactic lectures in six curriculum cycles and four in-person weekend workshops. Participants received 6004 h of no-cost medical education and claimed 3433 h of continued medical education (CME) credits. (Not all attendees required or requested CME credits). Participation in ECHO is cost free for HCPs because the project is permanently funded by the Ontario MOHLTC.

Study design

A pre-post online survey was used to evaluate the impact of ECHO in Ontario, Canada. The post-ECHO survey was sent by email when the subjects informed they were exiting the programme. The pre and post surveys were linked within subjects.

Participants and procedure

HCPs from across Ontario were recruited for participation in ECHO. Participants provided informed consent while completing the online questionnaire. The study was approved by the Research Ethics Board at the University Health Network (#14-7415).

Respondents were recruited to ECHO between June 2014 and March 2017. Participant eligibility included practicing in Ontario, the ability to speak and read English, and attendance of at least one ECHO session. We excluded participants who were still attending ECHO at the time of the study closure.

Measures

Participant demographics

We compiled information about participants’ age, gender, years in practice, profession and number of CP patients seen per week. Given the varied clinical backgrounds, we grouped attendees as those who can make decisions about opioid prescribing including physicians, physician assistants (PA) and nurse practitioners (NP); those who do not prescribe opioids but work closely with opioid prescribers: pharmacists and registered nurses (RN), and other allied health professionals such as psychologists, physiotherapists, occupational therapists, chiropractors and social workers.

We recorded the number of sessions each participant attended and whether or not they had presented a case during an ECHO session.

Outcome measures

Self-efficacy

We used a 19-item self-efficacy scale initially developed to assess the first ECHO Hepatitis C programme in New Mexico on a seven-point Likert scale ranging from 1 = ‘none or no skill’, to 7 = ‘expert, able to teach others’.¹⁷ We adapted the questions to assess participant’s confidence in CP management and opioid prescribing (See supplementary Appendix 1).

Knowledge

We assessed knowledge via eight items adapted from the KnowPain-12.²¹ Knowpain-12 is a newly developed tool aiming to assess provider’s pain management knowledge/attitude/behaviour. Since we were aiming to assess knowledge, we only selected the items that assess knowledge from Knowpain-12 and excluded others that are related to attitudes and behaviours. We also excluded one question that is specific to the United States healthcare system (See supplementary Appendix 1). Total scores could range from 0 to 16. After selecting the knowledge items, a pilot study was conducted to assess whether all the items were appropriate for non-physician participants or not. However, many of the non-physician participants expressed concerns that the knowledge test was supposed to be answered only by physicians and irrelevant to their field of practice. Many of the non-physician participants were leaving these questions blank. Therefore, the response options were also changed from the original version. The response options included ‘true’, ‘false’, ‘don’t know’ or ‘not applicable to my practice’.

Acceptability and impact of participation in ECHO

We used a self-report validated 11-item questionnaire developed by Arora et al. to measure acceptability and impact of ECHO.¹⁷ This questionnaire includes statements that assess value of participation, achievements of learning goals, professional satisfaction and isolation, subjective impression of quality of care, sense of participation in a community of practice, and perceptions of practice changes and variations in care. We rated each statement using a six-point Likert scale ranging from 1 = ‘Strongly Disagree’ to 5 = ‘Strongly Agree’, with 6 representing ‘Not Applicable’ (See supplementary Appendix 1).

Data analyses

Data analyses were performed using the latest version of Statistical Package for Social Sciences (SPSS 23.0). Demographic questionnaire response data were summarised using descriptive analyses (i.e. mean ± SD, range, percentage) to characterise the sample and variables of interest.

We examined differences between the study participants and dropouts via t-tests for the continuous variables (sessions attended, self-efficacy), and chi-square analyses and Fisher’s exact tests, for the binary and categorical variables (profession type and case presentation). Few missing data points were observed on individual items (less than 20% missing on individual items for each scale). These missing values were replaced using case mean substitution, which is considered to be a robust method of handling missing data when missingness is less than 30%.^22,23

Change scores for each outcome (self-efficacy and knowledge) were computed and differences between profession types were evaluated via analysis of variance (ANOVA). Repeated measures analysis of variance was used to assess differences in each main outcomes (self-efficacy and knowledge) before and after participation in ECHO by provider status and case presentation (yes or no). Pearson product moment or Spearman’s correlation coefficients estimated the relationship between changes in outcomes and the number of sessions attended by each participant.

Acceptability and Impact of ECHO was also assessed using chi-square tests for overall sample, and summarised by profession type and by case presentation.

All tests were two-tailed and a Bonferroni correction was performed on each set of analyses to control against Type 1 error. Effect size was calculated by dividing the average mean difference by the standard deviation for paired difference. Cohen suggests effect sizes of 0.2 or less are considered small, effect sizes of 0.5 are considered medium and 0.8 are considered large.²⁴ Cohen’s d effect sizes were also calculated for each paired difference.

Normality was also assessed by visually inspecting q-q residual and histogram plots, as well as by examining skewness and kurtosis values associated with each outcome variable, both for overall sample and by profession type and case presentation.

Results

Over the course of the study, 296 participants enrolled in ECHO (Figure 1). This study focuses only on these 119 eligible clinicians who completed both pre- and post-ECHO questionnaires. Data indicate that dropouts were more likely to have not presented a case during ECHO, but they were similar in terms of profession, location of practice, age, sex, level of education or pre-ECHO outcome measures.

Figure 1.

ECHO recruitment and dropouts. ECHO: extension for community healthcare outcomes.

Participant characteristics

ECHO participants represented a variety of professions (Table 1): 59% were prescribers or PAs and NPs who have an indirect influence on the prescriber they work with; 18% were pharmacists and RNs, and 23% other allied health professionals. The participants were primarily females (n = 89, 75%) with a mean age range from 40 to 49 years; 47% of participants were recent graduates new to clinical practice (between 1 and 9 years of practice).

Table 1.

Demographics.

	n	%
Professions	119	100
Physician/PA/NP (direct or indirect opioid prescribers)
Physician	41	34
NP	25	21
PA	4	3
Total	70	59
Pharmacist and nurses
Pharmacist	15	13
RN	7	6
Total	22	18
Allied health professions
Psychologist	1	<1
PT	2	<2
OT	6	5
Chiropractor	1	<1
Kinesiologist	1	<1
SW	12	10
Dietician	1	<1
Others	3	2
Total	27	23
Type of practice
Family health team	70	59
Community health centre	19	16
Hospital	11	9
CP clinic	11	9
Aboriginal health access centre	2	2
Solo practice	2	2
Mental health service centre	2	2
NP-led clinic	1	1
Fee-for-service practice	1	1
Sex
Female	89	75
Male	30	25
Age
20–29 years	12
30–39 years	32
40–49 years	33
50–59 years	24
60–69 years	8
70–79 years	2
Missing data	8
Years in practice
1–9 years	47
10–19 years	22
20–29 years	25
30–39 years	9
40–49 years	4
Missing data	12
Number of CP patients seen per week mean (SD)	12(12)

NP: nurse practitioner; PA: physician assistant; PT: physiotherapist; OT: occupational therapist; SW: social worker; CP: chronic pain.

Participants attended an average of 15 ± 9.19 2-h sessions (range 2–53). Sixty-six clinicians presented 196 cases: 164 new cases and 32 follow-ups. The most common pain diagnoses included low back pain, migraine headaches, neck pain, fibromyalgia, myofascial pain syndromes and neuropathies.

Self-efficacy

Of the 111 participants who responded to the self-efficacy pre- and post-ECHO questionnaires, 63 (58%) were Physicians/PA/NPs, 21 (19%) RN/Pharmacists and 25 (23%) allied health professionals.

We found increased confidence in pain management from pre- to post-testing, both for individual items and for the overall self-efficacy score. The total effect size for self-efficacy pre- to post-ECHO was 0.95, which is considered large.

Significant main effects for prescriber status (F(1, 111) = 50.842, p = .000, η² = .31) and time (F(1, 111) = 43.6, p = .000, η² = .51) were found suggesting that, on average, prescribers reported greater self-efficacy than non-prescribers and that self-efficacy scores increased significantly from pre- to post-ECHO (Figure 2).

Figure 2.

Self-efficacy scores pre and post-ECHO by (prescribers: physicians, NPs and PAs; non-prescribers: all other professions). PA: physician assistant, NP: nurse practitioner.

Further, a significant interaction effect was observed demonstrating that self-efficacy scores increased differently from pre to post-ECHO depending on prescriber status, F(1, 111) = 14.768, p = .000, η² = .12. Although simple main effects demonstrate that non-prescribers self-efficacy scores significantly increased from pre- to post-ECHO programme (p < .001), prescribers reported significantly greater self-efficacy on average following the ECHO programme than non-prescribers (F(1, 111) = 24.92, p = .000, η² = .18 (Figure 2).

No differences in self-efficacy between case-presenters and non-case-presenters were found after adjusting for type 1 error (F(1, 111) = 5.36, p = .022, η² = .046). There was no significant relationship between overall self-efficacy change and number of sessions attended (r = .175, p = .066).

Knowledge

A total of 105 participants responded to the knowledge section pre- and post-ECHO. Knowledge significantly increased for all ECHO participants, t(104) = −8.58, p = .000, d = .84.

Significant main effects for prescriber status (F(1, 104) = 15.32, p = .000, η² = .12) and time (F(1, 104) = 160.193, p = .000, η² = .58) were found, suggesting that prescribers had greater knowledge on average compared with non-prescribers and that knowledge increased post-ECHO programme. No significant interaction between prescriber status and time was observed, F(1, 104) = .272, p = .603, η² = .002 (Figure 3).

Figure 3.

Knowledge scores pre and post-ECHO by profession (prescribers: physicians, NPs and PAs; non-prescribers: all other professions).

Attending ECHO led to a significant increase in average knowledge scores in both case presenters and non-presenters, F(1, 104) = −105.95, p = .000, η² = .41). The increase in knowledge from the programme was similar between presenters and non-presenters, F(1, 104) = .016, p = .899, η² = .00), and no significant interaction between case presentation and time was demonstrated, F(1, 104) = .456, p = .501, η² = .004).

There was no significant relationship between knowledge change and number of sessions attended, correlation coefficient, r_s = .08, p = .407, or t-tests stratified by number of sessions attended (21 or more sessions), t(103) = −.381, p = .704, d = 0.09.

Acceptability and impact of participation in ECHO

Of the 117 participants who responded to the acceptability and impact section post-ECHO, the majority agreed or strongly agreed with the 11 impact statements (Figure 4); 96% of participants said they would recommend ECHO to any of their colleagues.

Figure 4.

Percentages of participants who selected agree/ strongly agree on the 11-item impact and acceptability scale.

Acceptability levels were equivalent between professions (p_s = .130 −.888). There was a significant positive relationship between number of sessions attended and three items (p-values ranged from .000 to .005).

Participants who presented a case in session were more likely to strongly agree with 3 of the 11 acceptability and impact items compared with participants who did not present a case (p-values ranged from .000 to .004).

Discussion

We evaluated the first 2.5 years of its implementation and assessed the impact of ECHO on participants’ self-efficacy, knowledge regarding CP management and opioid prescribing, along with participants’ acceptability and impact after attending ECHO.

Self-efficacy is a construct described by Albert Bandura,^25,26 and is defined as the individual’s belief in their capacity to execute a behaviour necessary to produce a specific action. It reflects the person’s ability to control their own motivation, behaviour and social environment. Gaining self-efficacy is the first step towards behaviour change.^27,28 Our findings demonstrated a significant increase in self-efficacy after attending ECHO. All professions improved their self-efficacy, but the groups of physicians, PAs and NPs had the highest gain in self – efficacy compared with other professions. These findings were expected since most of the didactic sessions and case presentations involve some sort of pharmacological treatment. Professionals who are not directly involved in medication management may not perceive their improvements the same as the professionals who are involved with medications more often.

Our results also showed that ECHO significantly improved participants' knowledge regarding CP assessment, treatment practices and opioid prescribing. Our results suggest that the ECHO intervention successfully enhanced provider knowledge of pain management regardless of profession. The ECHO model applies adult learning theory when participants present cases from their own practices, so new teaching and learning are applied at the right time and the right place. The recommendations on each case presented are discussed in an informal discussion by the whole group, where all teach and all learn. These improvements in clinician’s knowledge are consistent with other ECHO replications in other health conditions such as mental health,²⁹ and hepatitis C.³⁰

Our attendees were highly satisfied with ECHO and felt that their participation had an impact in improving the care they offer to their clients with chronic pain. Participation in ECHO was a worthwhile experience that enhanced their professional satisfaction and reduced isolation. Though no differences were found in acceptability levels between different professions groups, participants who presented a case were more satisfied than those who did not present cases. Weekly case-based learning is a critical component of the ECHO model and is an effective method in continuing medical education pedagogy.^17,31,32

Our attendees also reported that attending ECHO impacted their practices to a great extent. The majority reported that attending ECHO improved the quality and safety of patient care for CP and has reduced variation in care and, most importantly, has expanded access to effective CP treatments in their community.³³

Our findings are consistent with others who have demonstrated that clinicians’ self-efficacy and knowledge can be improved by ECHO.^14,34 A recent systematic review by Zhou and colleagues found that ECHO is an effective and potentially cost-saving model that increased clinicians’ self-efficacy, confidence, changed their attitudes and improved their knowledge with subsequent improvements in patients’ outcomes. This was especially true for HCPs who are working in isolated rural locations.³⁵ This longitudinal learning process may promote direct uptake in practice since the clinicians often return to clinical activities shortly after the completion of each weekly session.

Our study’s strength is a pre-post intervention study design where subjects served as their own controls. We evaluated HCPs’ outcomes from a multidisciplinary group of clinicians practicing in Ontario. This diversity helps with the generalisability of our findings. In addition, we were able to recruit and retain many opioid prescribers to attend ECHO, and it seems that this group was the one that gained most from their participation in ECHO.

Despite our efforts to ensure rigour and reduce bias, one limitation of this study is that the pre-test may sensitise participants to the intervention and change the way that they respond to the post-test.³⁶ Moreover, some may argue that all our outcome measures are self-reported and purely subjective measures leaving many unanswered questions, such as: how ECHO works in reducing professional isolation, improving joy of work and patients’ outcomes. Nevertheless, although several studies assessing the impact of ECHO have shown a positive correlation between self-reported changes and patients outcomes or changes in attendees’ practices, we are still aware of this limitation. Therefore, we are currently conducting other studies using objective measures. Ongoing research include mixed methods such as qualitative interviews with clinicians and patients, and quantitative methods to assess practice changes that would reflect patients’ receiving appropriate investigations, education, treatment and management of their CP by their primary care providers. We think it is important to also demonstrate if ECHO has the capability to reduce wait lists to specialist care, and reduce variation and costs of health care.

Conclusions

The ECHO model developed at the University of New Mexico was successfully implemented in Ontario to address a knowledge and practice gap in CP management and opioid prescribing. Participation in ECHO was associated with significant improvement in self-efficacy and knowledge about CP for all participants, but it was more pronounced in participants who can prescribe opioids than allied professionals who do not prescribe medications. Participants who present cases during ECHO sessions are more satisfied with ECHO than those who only attend and do not present cases. On average participants attended 30 h (range 4–106 h) of ECHO sessions, but improvements were not dose dependent.

Supplemental Material

Appendix -Supplemental material for Evaluation of an innovative tele-education intervention in chronic pain management for primary care clinicians practicing in underserved areas

Supplemental material, Appendix for Evaluation of an innovative tele-education intervention in chronic pain management for primary care clinicians practicing in underserved areas by Andrea D Furlan, Jane Zhao, Jennifer Voth, Samah Hassan, Ruth Dubin, Jennifer N Stinson, Susan Jaglal, Ralph Fabico, Andrew J Smith, Paul Taenzer and John F Flannery in Journal of Telemedicine and Telecare

Footnotes

Acknowledgements

The authors report a research grant from Canadian Institutes for Health Research (PHE-201404-318808), and an operational award from Ontario Ministry of Health and Long-Term care during the conduct of the study. This work has been presented as a ‘poster’ at the following scientific meetings: 16^th World Congress on Pain (IASP) Yokohama, Japan, 30 September 2016 and 38th Annual Scientific Meeting, Canadian Pain Society, Halifax NS, 24 May 2017.

Declaration of conflict of interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental material

Supplemental material for this article is available online.

References

Moulin

Clark

Speechley

, et al. Chronic pain in Canada-prevalence, treatment, impact and the role of opioid analgesia. Pain Res Manag 2002; 7: 179–184.

Schopflocher

Taenzer

Jovey

. The prevalence of chronic pain in Canada. Pain Res Manag 2011; 16: 445–450.

Johannes

Zhou

, et al. The prevalence of chronic pain in United States adults: results of an Internet-based survey. J Pain 2010; 11: 1230–1239.

Vos

Flaxman

Naghavi

, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; 380: 2163–2196.

Lynch

. The need for a Canadian pain strategy. Pain Res Manag 2011; 16: 77–80.

Andrew

Derry

Taylor

, et al. The costs and consequences of adequately managed chronic non-cancer pain and chronic neuropathic pain. Pain Pract 2014; 14: 79–94.

Dowell

Haegerich

Chou

. CDC guideline for prescribing opioids for chronic pain—United States, 2016. JAMA 2016; 315: 1624–1645.

Watt-Watson

Murinson

. Current challenges in pain education. Pain Manag 2013; 3: 351–357.

Watt-Watson

McGillion

Hunter

, et al. A survey of prelicensure pain curricula in health science faculties in Canadian universities. Pain Res Manag 2009; 14: 439–444.

10.

Gomes

Juurlink

Moineddin

, et al. Geographical variation in opioid prescribing and opioid-related mortality in Ontario. Healthc Q 2011; 14: 22–24.

11.

Chou

Turner

Devine

, et al. The effectiveness and risks of long-term opioid therapy for chronic pain: A systematic review for a National Institutes of Health Pathways to Prevention Workshop. Ann Intern Med 2015; 162: 276–286.

12.

Lalonde

Leroux-Lapointe

Choinière

, et al. Knowledge, attitudes and beliefs about chronic noncancer pain in primary care: a Canadian survey of physicians and pharmacists. Pain Res Manag 2014; 19: 241–250.

13.

Allen

Asbridge

Macdougall

, et al. Self-reported practices in opioid management of chronic noncancer pain: a survey of Canadian family physicians. Pain Res Manag 2013; 18: 177–184.

14.

Katzman

Comerci

Boyle

, et al. Innovative telementoring for pain management: Project ECHO pain. J Contin Educ Health Prof 2014; 34: 68–75.

15.

Shelley

Katzman

Comerci

Jr , et al. ECHO pain curriculum: Balancing mandated continuing education with the needs of rural health care practitioners. J Contin Educ Health Prof 2017; 37: 190–194.

16.

Frank

Carey

Fagan

, et al. Evaluation of a telementoring intervention for pain management in the Veterans Health Administration. Pain Med 2015; 16: 1090–1100.

17.

Arora

Kalishman

Thornton

, et al. Expanding access to hepatitis C virus treatment—Extension for Community Healthcare Outcomes (ECHO) project: disruptive innovation in specialty care. Hepatology 2010; 52: 1124–1133.

18.

Arora

Thornton

Komaromy

, et al. Demonopolizing medical knowledge. Acad Med 2014; 89: 30–32.

19.

Komaromy

Duhigg

Metcalf

, et al. Project ECHO (Extension for Community Healthcare Outcomes): a new model for educating primary care providers about treatment of substance use disorders. Subst Abus 2016; 37: 20–24.

20.

Dubin

Flannery

Taenzer

, et al. ECHO Ontario chronic pain & opioid stewardship: Providing access and building capacity for primary care providers in underserviced, rural, and remote communities. Stud Health Technol Inform 2015; 209: 15–22.

21.

Gordon

Loeser

Tauben

, et al. Development of the KnowPain-12 pain management knowledge survey. Clin J Pain 2014; 30: 521–527.

22.

El-Masri

Fox-Wasylyshyn

. Missing data: an introductory conceptual overview for the novice researcher. Can J Nurs Res 2005; 37: 156–171.

23.

Raymond

. Missing data in Evaluation Research. Eval Health Prof 1986; 9: 395–420.

24.

Cohen

. Statistical power analysis for the behavioral sciences, (revised ed.). New York: Academic Press, 1977.

25.

Bandura

. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 1977; 84: 191–215.

26.

Bandura

Adams

Beyer

. Cognitive processes mediating behavioral change. J Pers Soc Psychol 1977; 35: 125–139.

27.

Hughes

Galbraith

White

. Perceived competence: a common core for self-efficacy and self-concept? J Pers Assess 2011; 93: 278–289.

28.

Arora

Thornton

Murata

, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. New Engl J Med 2011; 364: 2199–2207.

29.

Sockalingam S, Arena A, Serhal E, et al. Building provincial mental health capacity in primary care: An evaluation of a project ECHO mental health program. Acad Psychiatry. 2018; 42(4): 451–457.

30.

Marciano

Haddad

Plazzotta

, et al. Implementation of the ECHO(R) telementoring model for the treatment of patients with hepatitis C. J Med Virol 2017; 89: 660–664.

31.

Thistlethwaite

Davies

Ekeocha

, et al. The effectiveness of case-based learning in health professional education. A BEME systematic review: BEME Guide No. 23. Med Teach 2012; 34: e421–e444.

32.

Scallan

Davis

Thomas

, et al. Supporting peer learning networks for case-based learning in public health: Experience of the Rocky Mountain Public Health Training Center with the ECHO training model. Pedagogy Health Promot 2017; 3: 52S–58S.

33.

Carlin

Zhao

Dubin

, et al. Project ECHO telementoring intervention for managing chronic pain in primary care: Insights from a qualitative study. Pain Med 2017; 19: 1140–1146.

34.

Ball

Wilson

Ober

, et al. SCAN-ECHO for pain management: Implementing a regional telementoring training for primary care providers. Pain Med 2017; 19: 262–268.

35.

Zhou

Crawford

Serhal

, et al. The impact of project ECHO on participant and patient outcomes: a systematic review. Acad Med 2016; 91: 1439–1461.

36.

Stommel M and Wills C. Clinical research: Concepts and principles for advanced practice nurses. Lippincott Williams & Wilkins, 2004.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB