Diagnostic and treatment concordance in primary care participants and dermatologists utilizing Extension for Community Health Outcomes (ECHO)

Abstract

Introduction

Suboptimal access to dermatologic care is dependent on patient location and insurance type. Although there have been attempts to address access issues, barriers to providing excellent dermatologic care to all patients at the right time still exist. The objective of this study was to investigate the clinical impact of Dermatology Extension for Community Healthcare Outcomes (ECHO) project participation on primary care providers’ diagnostic and treatment tendencies and accuracy.

Methods

This was a retrospective cohort study constructed using Dermatology Extension for Community Healthcare Outcomes case and recommendation data from November 2015 to June 2021. The University of Missouri-based Dermatology Extension for Community Healthcare Outcomes specialty hub team offers regularly scheduled live interactive tele-mentoring sessions for primary care providers who practice in rural and underserved areas. 524 patient cases presented by 25 primary care providers were included in the analysis. Of those, 449 cases were included in diagnostic concordance, and 451 in treatment concordance analysis.

Results

Less than 40% of all diagnoses were fully concordant with an expert panel. Over 33% of patients were misdiagnosed, and over 26% received partially correct diagnosis. Only 16% of all treatment recommendations were fully concordant with an expert panel.

Discussion

Diagnostic and treatment accuracy of participants is low, and Dermatology Extension for Community Healthcare Outcomes platform ensured patients received correct diagnosis and treatment quickly. Although tele-dermatology models are effective, they continue to be underutilized. Dermatologists in practice and training should be encouraged to adopt innovative clinical educational models, like Dermatology ECHO, to expand access to dermatologic expertise for the most marginalized populations.

Keywords

Telehealth teleconsulting teledermatology tele-education Extension for Community Healthcare Outcomes Project ECHO

Introduction

Dermatologists are failing to meet the skin health needs of large populations of patients in the United States.^1–4 The American Academy of Dermatology (AAD)'s most recent Burden of Skin Disease report shows that one in four individuals (aged 0–64 years.) have skin disease treated each year, as do half of those 65 years. and older.⁵ Two-thirds of patients with skin disease are treated by health care providers who are not board-certified dermatologists and part of the reason for this appears to be barriers to referral.⁵ Recent surveys through AAD's Specialty Positioning initiative demonstrate that one-third to one-half of non-dermatology physicians report having serious difficulty referring patients to dermatology when needed.⁶

Suboptimal access to dermatologic services may be highly dependent on patient insurance type.^4,7,8 Medicaid acceptance rates by dermatologists for patients of all ages in 2004 was 32%.^9,10 In addition, only 17% of nonacademic dermatologists accept Medicaid.¹¹ A more recent study reported that privately insured patients are accepted by dermatologists over 90% of the time, compared to 29.8% of publicly insured patients.¹² Access to pediatric dermatologists is even more limited, reflected by a 44% average refusal rate of “new pediatric Medicaid-insured patients by Medicaid-listed providers” and the overall pediatric Medicaid acceptance rate of only 19%.⁷ Lack of access to dermatologic expertise is detrimental to patients, resulting in greater morbidity and mortality.^13–15 The impact of skin disease on patients’ mental health is also well established.^16–19

There have been multiple attempts to address the problem of access to dermatologic services but it continues to be a barrier for the specialty in meeting its strategic goal of providing excellent dermatologic care at the right time and place for all patients.²⁰ There are different contributing factors influencing accessibility, including inadequate numbers of board-certified dermatologists relative to population, preference for practice in urban or suburban locations, and practices limiting the number of Medicaid or uninsured patients they accept.²¹ The solution goes well beyond simply training more dermatologists. Creative and innovative practices are needed today whereby dermatologic expertise can be exported to those providing care in dermatologically underserved areas.

Project Extension for Community Healthcare Outcomes (ECHO)

Project ECHO is an innovative educational and mentoring model that began 16 years ago as the brainchild of Sanjeev Arora, MD, a board-certified hepatologist at the University of New Mexico.^22–24 Dr. Arora identified an unacceptably high number of patients in his state dying of untreated hepatitis C secondary to poor access to care and specialty expertise, as demonstrated by a 10-month wait to be seen in his clinic. The program he developed is based on a Hub and Spoke model of educational conferencing where primary care providers (PCPs) and other clinicians (Spokes) videoconference with an expert (Hub) team using web conferencing technology from their clinical setting or home.^25,26 The hub team is multidisciplinary, and the conference consists of a short didactic followed by discussions about patient cases that are challenging for the PCPs. It creates a collegial, supportive, tele-mentoring learning collaborative where evidence-based, best practices are shared and discussed. Rates of sustained virologic response of patients treated by PCPs participating in Hepatitis C ECHO (58.2%) were similar to those treated at an academic medical center (49.7%), showing that application of ECHO model resulted in high cure rates.²³ This case-based format is founded on the principles of adult learning and widely accepted as a more effective educational approach than traditional continuing medical education (CME) formats.^27–29

There are currently 1117 ECHO programs in the United States, and over 1000 in 60 countries worldwide.³⁰ They focus on different diseases and conditions, such as autism, child psychiatry, COVID-19, and opioid use disorder, or community-related topics, such as public health, collaborative education, adult vaccination, among others. ^30–33

Dermatology ECHO

The first Dermatology ECHO in the world began in 2015 at the University of Missouri (MU) with a hub team of experts that includes general and pediatric dermatology and dermatopathology.^34,35 Since then, 16 Dermatology ECHO programs have been developed in seven countries.³⁰

The MU Dermatology ECHO biweekly hour-long sessions Consist of a hub team expert didactic presentation and one or more de-identified case presentation(s) by participating PCPs.^35–38 Short introductions are followed by a 10 to 15-minute expert didactic presentation related to skin disease. Previous presentations included topics such as delusions of parasitosis, dysplastic nevi, evaluation of pigmented lesions, melanoma screening and detection, systemic therapy for psoriasis, among others. A Dermatology ECHO hub team clinical lead moderates case presentations and discussions that follow the didactic presentation.

Participating PCPs submit de-identified case presentations 3–5 days prior to the Dermatology ECHO. PCPs who are interested in presenting a case complete a short case form that includes patient age, gender, preliminary diagnosis (if any), preliminary treatment (if any), history of the skin condition, and de-deidentified pictures of the condition are enclosed. The form also requests the impetus for case presentation, such as questions about diagnosis or treatment. Cases are uploaded into the Microsoft OneDrive protected cloud storage, and reviewed by the hub team prior to the Dermatology ECHO session.³⁹ Two to nine cases are typically discussed during each hour-long session.

During a 3 to 5-minute PCP-led case discussion, submitted images are shared with participants by the hub team clinical lead. Case presentations are followed by clarifying questions asked by hub team members and other participants. The hub team clinical lead invites hub team members to provide their expert opinion and recommendations regarding the case. Dermatology ECHO supports the “all teach, all learn” learning framework, and the clinical hub team lead facilitator also invites participating PCPs to share their experiences and provide any additional recommendations regarding the case.⁴⁰ After each case discussion, the clinical hub team lead facilitator completes the “recommendation form” summarizing recommendations, which is shared with participants using the OneDrive folder.

Occasionally participants may disagree when discussing case recommendations, although this is a rare phenomenon. When it does occur, the clinical hub team lead will summarize all recommendations made during the discussion and the final decision on diagnosis and treatment will be at the discretion of PCP presenting the case. In this case the hub team will also ask the PCP to present the follow-up outcomes discussion in the future and share lessons learned with all participants.

Dermatology ECHO sessions are recorded, with recordings used only for research and evaluation purposes. Program evaluation shows an increase in participating PCPs’ self-efficacy in diagnosing and treating a variety of basic skin diseases.^41,42

Dermatology ECHO offers the promise of greater clinical understanding and quality of care by enhancing communication between participating PCPs and dermatologists. The primary goal of this study was to investigate the clinical impact of Dermatology ECHO on participating PCPs’ diagnostic and treatment choices and accuracy.

Materials and methods

The MU Institutional Review Board (IRB) determined that this project included de-identified anonymous secondary data analysis, therefore not requiring IRB approval.

This was a retrospective cohort study constructed using Dermatology ECHO case data from November 2015 to June 2021. Cases from 19 November 2015 to 3 January 2020, were submitted on PDF forms, and cases submitted after 4 January 2020, were submitted through an online Research Electronic Data Capture (REDCap)⁴³ form and exported into Excel.⁴⁴ All case data was then combined and entered in an Excel database.⁴⁴

Dermatology ECHO case presentations

A total of 543 cases were presented during the study period. After removing cases with missing matching preliminary and final diagnostic or treatment data, 524 cases were included in the analysis. Of those, 449 cases were included in diagnostic concordance analysis, and 451 in treatment concordance analysis. Nine data elements were extracted from each presented case: patient age, patient race, patient sex, PCPs’ clinic county, unique patient count, preliminary diagnosis, hub team recommended diagnosis, preliminary treatment, and hub team recommended treatment.

Preliminary diagnosis and treatment presented by PCPs were then matched with final hub team recommended diagnosis and treatment. Participating PCPs were 15 physicians—medical doctors (MDs) and Doctor of Osteopathic Medicine (DOs) and 10 nonphysician providers—advance practice nurses (APRNs) and physician assistants (PAs).

Concordance analysis considered the hub team recommended diagnosis and treatment as accurate and final. The following concordance definitions were used: 0 = poor, 0.5 = fair, and 1 = perfect (Table 1). Two dermatologists (KE and KB) independently coded diagnostic and treatment concordance. Study co-author (MB) reviewed and reconciled cases and identified coding differences. KE, KB, and MB met and corrected coding disagreements. Through discussion, each dermatologist shared their rationale for coding and came to a unified conclusion as to the final coding determination.

Table 1.

Example concordance coding.

Preliminary diagnosis	Final diagnosis	Diagnostic concordance
Granulomatous dermatitis	Granulomatous dermatitis	1
Lupus versus Sjogren's versus melasma versus postinflammatory hyperpigmentation versus lentigines	Melasma	0.5
Sebaceous gland hyperplasia	Perioral derm/acne	0

Preliminary treatment	Final treatment		Treatment concordance
None	Monitor. Can take out later if grows and becomes bothersome		1
PCP—oral prednisone, oral doxy, and topical clindamycin.Currently on protopic x1 week, no perceived improvement.	Doxycycline twice a day and Plaquenil for some weeks. If not improve, do a punch biopsy		0.5
Remicade, IVIG and re-biopsy	Rituximab (Rituxan)		0

Diagnostic and treatment concordance were calculated with 95% confidence intervals (CIs). The chi-square test for Equal Proportions was used to investigate the equality of completeness levels.

Results

Dermatology ECHO case presentations

A total of 25 unique participating PCPs presented 524 cases over a seven-year period (Table 2). We observed that the frequency of sessions changed in 2016, when weekly Dermatology ECHO sessions transitioned into twice monthly, reducing the number of annual sessions. In addition, 2015 data includes only two months of ECHO sessions (November and December), and 2021 data included 6 months of ECHO sessions (January–June).

Table 2.

Case presentations per year.

Year	Case presentations (n)	Percent	Cumulativefrequency	Cumulativepercent	Sessions (n)
2015	3	0.57	3	0.57	4
2016	66	12.60	69	13.17	35
2017	157	29.96	226	43.13	39
2018	138	26.34	364	69.47	34
2019	86	16.41	450	85.88	21
2020	56	10.69	506	96.56	19
2021	18	3.44	524	100.00	8

Of the presented cases, 81% (n = 382) were adult, and 19% (n = 88) were pediatric cases. Mean age of patients was 44.03. We observed 54 patient cases missing age information.

Presenting PCPs

A total of 25 PCPs presented cases during the study, 22 from MU, and one each from Texas and Kentucky; one clinician did not indicate location. The average number of case presentation per PCP was 20.96, with median of seven.

Diagnostic trends and concordance

Table 3 shows the summary of diagnostic concordance for all cases. The chi-square test for Equal Proportions indicated the proportions is statistically significantly different (p < 0.0001). The percentage of complete agreement (39.87%) is significantly higher than partial agreement (26.73%) and no agreement (33.41%). The percent of complete diagnostic concordance is similar over the years (Figure 1).

Figure 1.

Diagnostic concordance by year.

Table 3.

Diagnostic and treatment concordance (n = 449).

Concordance definitions	Frequency	Percent	95% confidence limit
0	150	33.41	29.04–37.77
0.5	120	26.73	22.63–30.82
1	179	39.87	35.34–44.56
Frequency missing = 75
Treatment concordance (n = 451)
Concordance definitions	Frequency	Percent	95% confidence limit
0	181	40.13	35.61–44.66
0.5	202	44.79	40.20–49.38
1	68	15.08	11.78–18.38
Frequency missing = 73

Treatment trends and concordance

Table 3 shows the summary of treatment concordance for all cases. The chi-square test for Equal Proportions indicated the proportion is statistically significant difference (p < 0.0001). The percentage of complete agreement is significantly lower than the percentage of partial agreement (16% vs. 44.50%) (Figure 2).

Figure 2.

Treatment concordance by year.

Diagnostic categories

Diagnostic categories presented in the Burden of Skin Disease report were used to sort and quantify Dermatology ECHO case presentations.⁵ A total of 449 unique diagnoses in 24 separate diagnostic categories were presented (Figure 3). Most cases were in the connective tissue category (n = 51), followed by contact dermatitis (n = 44), and psoriasis/lichen planus (n = 41) (Figure 3).

Figure 3.

Dermatology Extension for Community Healthcare Outcomes (ECHO) diagnostic categories.

Conclusion

Patient access to healthcare varies greatly and is influenced by geolocation, health insurance status, and socioeconomic status.⁴⁵ For dermatology, the workforce shortage remains geographically imbalanced, disproportionately affecting access for patients in rural and other underserved areas that have not seen significant transformative improvement for decades.^46–48 Although tele-dermatology has the potential to improve access and reduce costs, barriers remain.^49,50 Absence of adequate broadband and connectivity, low reimbursement, liability concerns, lack of access to and knowledge of local resources, and local absence of procedural care have prevented implementation and adoption of telehealth technologies across service regions and healthcare systems.^51–55

It is estimated that over 50% of the US population has some form of skin disease, of which 22.5% is moderate or severe.^42,56,57 Lowell et al. reported that in primary care settings, almost 37%of patients present with at least one skin problem over a 2-year period.⁵⁸ However, studies have shown lack of adequate training and education of PCPs, both physician and nonphysician, in diagnosis and treatment of skin disease often result in delayed or missed diagnosis and costly treatments.^59,60 Most skin lesions are first identified in primary care.^59–61 Diagnostic concordance data comparing accuracy of PCPs and dermatologists in identifying melanoma reveals that PCPs achieve lower diagnostic accuracy compared to dermatologists.⁵⁹ However, continuing education, mentoring, and collaboration with dermatologists may help PCPs triage suspicious lesions with greater precision.⁶² This study included all cases presented in Dermatology ECHO, malignant and nonmalignant lesions and conditions. While complete diagnostic agreement (39.87%) between participating PCPs and dermatologists was higher than partial agreement (26.73%) or no agreement (33.41%), less than 40% of all diagnoses were fully concordant with an expert panel. Over 33% of patients were misdiagnosed while 26.73% received partially correct diagnoses. Over 60% of patients received preliminary diagnoses that were partially or completely in disagreement with dermatologists.

Though diagnostic and treatment accuracy was low, the patients presented in Dermatology ECHO ultimately received timely and correct diagnosis and treatment. Expert hub team members have also noticed cases presented from long-standing participants tend to become more complex and less common over time.

Recommendation

Dermatologists in practice and training should be encouraged to adopt innovative clinical educational models like Dermatology ECHO to expand access to dermatologic expertise, especially for geographically and socioeconomically underserved patient populations, thus providing expert dermatologic care at the right time and place for more patients.^1,20 Although tele-dermatology models, including ECHO, are effective, they continue to be underutilized, emphasizing the need for implementation of innovative, multipronged strategies.¹

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Health Resources and Services Administration (grant number SP13977-SB11).

ORCID iDs

Mirna Becevic

Chi-Ren Shyu

References

Pearlman

Brodell

Byrd

. Enhancing access to rural dermatological care: The time to start is now. JAMA Dermatol 2022; 158: 725–726.

Association between the use of a physician extender and dermatology appointment wait times in Ohio. Dermatology. JAMA Dermatology. JAMA Network [Internet], https://jamanetwork.com/journals/jamadermatology/fullarticle/2653790 (2017, accessed 17 August 2022).

Privalle

Havighurst

Kim

, et al. Number of skin biopsies needed per malignancy: Comparing the use of skin biopsies among dermatologists and nondermatologist clinicians. J Am Acad Dermatol 2020; 82: 110–116.

Cortez

Vasquez

Wei

. The impact of demographics, socioeconomics, and healthcare access on melanoma outcomes. J Am Acad Dermatol 2020. https://www.sciencedirect.com/science/article/pii/S0190962220324051 (accessed 29 March 2021).

Lim

Collins

SAB

Resneck

, et al. The burden of skin disease in the United States. J Am Acad Dermatol 2017; 76: 958–972.e2.

AAD VMX 2021 [Internet], https://www.aad.org/member/meetings-education/aadvmx (2021, accessed 24 May 2021).

Pediatric access to dermatologists: Medicaid versus private insurance - ClinicalKey [Internet], https://www.clinicalkey.com/#!/content/playContent/1-s2.0-S0190962212011693?scrollTo=%23hl0000720 (2013, accessed 28 July 2022).

Cortez

Fadadu

Konda

, et al. Disparities in access for melanoma screening by region, specialty, and insurance: A cross-sectional audit study. JAAD Int 2022; 7: 78–85.

Resneck

Pletcher

Lozano

. Medicare, Medicaid, and access to dermatologists: The effect of patient insurance on appointment access and wait times. J Am Acad Dermatol 2004; 50: 85–92.

10.

Creadore

Desai

, et al. Insurance acceptance, appointment wait time, and dermatologist access across practice types in the US. JAMA Dermatol 2021; 157: 181–188.

11.

Williams

Amerson

Chang

. How dermatologists can address the structural and social determinants of health—from awareness to action. JAMA Dermatol 2022; 158: 351–352.

12.

Alghothani

Jacks

Vander Horst

, et al. Disparities in access to dermatologic care according to insurance type. Arch Dermatol 2012; 148: 956–957.

13.

Aneja

Bordeaux

. Association of increased dermatologist density with lower melanoma mortality. Arch Dermatol 2012; 148: 174–178.

14.

Eide

Weinstock

Clark

. The association of physician-specialty density and melanoma prognosis in the United States, 1988 to 1993. J Am Acad Dermatol 2009; 60: 51–58.

15.

DeFazio

Marghoob

Pan

, et al. Variation in the depth of excision of melanoma: A survey of US physicians. Arch Dermatol 2010; 146: 995–999.

16.

Schuster

Ziehfreund

Albrecht

, et al. Happiness in dermatology: A holistic evaluation of the mental burden of skin diseases. J Eur Acad Dermatol Venereol 2020; 34: 1331–1339.

17.

Stanescu

AMA

Totan

Mircescu

, et al. Assessment of suicidal behavior in dermatology (Review). Exp Ther Med 2020; 20: 73–77.

18.

Nasimi

Ahangari

Lajevardi

, et al. Quality of life and mental health status in patients with lichen planopilaris based on Dermatology Life Quality Index and General Health Questionnaire-28 questionnaires. Int J Women’s Dermatol 2020; 6: 399–403.

19.

Russell

Bennett

Ali

, et al. Identifying mental health needs of children and youth with skin disease: A systematic review of screening and assessment tools. Pediatr Dermatol 2022; 39, https://onlinelibrary.wiley.com/doi/abs/10.1111/pde.14991 (2022, accessed 28 July 2022).

20.

Strategic plan [Internet], https://www.aad.org/member/membership/governance/strategic (2022, accessed 12 August 2022).

21.

Ehrlich

Kostecki

Olkaba

. Trends in dermatology practices and the implications for the workforce. J Am Acad Dermatol 2017; 77: 746–752.

22.

Arora

Thornton

Komaromy

, et al. Demonopolizing medical knowledge. Acad Med 2014; 89: 30–32.

23.

Arora

Thornton

Murata

, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med 2011; 364: 2199–2207.

24.

Arora

Kalishman

Thornton

, et al. Project ECHO (project extension for community healthcare outcomes): A national and global model for continuing professional development. J Contin Educ Health Prof 2016; 36: S48–S49.

25.

Arora

Smith

Snead

, et al. Project ECHO: An effective means of increasing palliative care capacity. Am J Manag Care 2017; 23: SP267–SP271.

26.

Michael Lewiecki

Bouchonville

Chafey

, et al. Telementoring with bone health ECHO to improve osteoporosis care in underserved areas. J Clin Densitom 2016; 19: 539–540.

27.

Reed

Shell

Kassis

, et al. Applying adult learning practices in medical education. Curr Probl Pediatr Adolesc Health Care 2014; 44: 170–181.

28.

Arora

Kalishman

Thornton

, et al. Project ECHO: A telementoring network model for continuing professional development. J Contin Educ Health Prof 2017; 37: 239–244.

29.

Katzman

Comerci

Boyle

, et al. Innovative telementoring for pain management: Project ECHO pain. J Contin Educ Health Prof 2014; 34: 68–75.

30.

Hubs & Programs [Internet], https://hsc.unm.edu/echo/partner-portal/data-marketplace/interactive-dashboards/ (2022, accessed 1 November 2022).

31.

Zhao

Rozenberg

Nourouzpour

, et al. Positive impact of a telemedicine education program on practicing health care workers during the COVID-19 pandemic in Ontario, Canada: A mixed methods study of an Extension for Community Healthcare Outcomes (ECHO) program. J Telemed Telecare 2021: 1357633X211059688.

32.

Scott

Unruh

Catlin

, et al. Project ECHO: A model for complex, chronic care in the Pacific Northwest region of the United States. J Telemed Telecare 2012; 18: 481–484.

33.

Koenig

Wenger

Graham

, et al.

Managing professional knowledge boundaries during ECHO telementoring consultations in two Veterans Affairs specialty care liver clinics: A theme-oriented discourse analysis

. J Telemed Telecare 2019; 25: 181–189.

34.

Lewis

Becevic

Myers

, et al. Dermatology ECHO - an innovative solution to address limited access to dermatology expertise. Rural Remote Health 2018; 18: 1–8.

35.

Bridges

Morris

McElroy

, et al. Utility of dermatology extension for community healthcare outcomes (ECHO) sessions in the adult and paediatric population. J Telemed Telecare 2021; 27: 376–381.

36.

Dermatology ECHO: A case presentation demonstrating benefits of specialty telementoring in primary care - Ryan Ladd, Mirna Becevic, Hope Misterovich, Karen Edison, https://journals.sagepub.com/doi/full/10.1177/1357633X18780562 (2019, accessed 17 August 2022).

37.

Becevic

Greever-Rice

Wallach

, et al. Assessing impact of show-me ECHO on the health of missourians: Two examples. Mo Med 2020; 117: 245–253.

38.

Warne-Griggs

Brandt

Hoffman

, et al. Building regional expertise: Perspectives from show-me ECHO participants. Mo Med 2020; 117: 222–227.

39.

How OneDrive safeguards your data in the cloud [Internet], https://support.microsoft.com/en-us/office/how-onedrive-safeguards-your-data-in-the-cloud-23c6ea94-3608-48d7-8bf0-80e142edd1e1 (2022, accessed 28 October 2022).

40.

The ECHO Model is a learning framework that results in sustainable change. [Internet], https://hsc.unm.edu/echo/what-we-do/about-the-echo-model.html (2022, accessed 28 October 2022).

41.

News [Internet]. Show-Me ECHO, h ttps://showmeecho.org/news/ (2017, accessed 17 August 2022).

42.

Sun

Green

Zaenglein

, et al. Efficacy of pediatric dermatology Extension for Community Healthcare Outcomes (ECHO) sessions on augmenting primary care providers’ confidence and abilities. Pediatr Dermatol 2022; 39: 385–388.

43.

REDCap [Internet], https://www.project-redcap.org/ (accessed 9 August 2021).

44.

Microsoft Excel Spreadsheet Software. Microsoft 365 [Internet], https://www.microsoft.com/en-us/microsoft-365/excel (2022, accessed 12 September 2022).

45.

Allen

Chatfield

Burleson

, et al. Improving diagnosis in health care: Perspectives from the American college of radiology. Diagnosis 2017; 4: 113–124.

46.

Green

Murchie

Lee

. Does Patients’ place of residence affect the type of physician performing primary excision of cutaneous melanoma in northern Scotland? J Rural Health 2013; 29: s35–s42.

47.

Coory

Smithers

Aitken

, et al. Urban-rural differences in survival from cutaneous melanoma in queensland. Aust N Z J Public Health 2006; 30: 71–74.

48.

Shah

Yang

Maverakis

, et al. Assessing rural–urban disparities in the use of sentinel lymph node biopsy for melanoma. J Surg Res 2013; 184: 1157–1160.

49.

Thomas

Haydon

Mehrotra

, et al. Building on the momentum: Sustaining telehealth beyond COVID-19. J Telemed Telecare 2022; 28: 301–308.

50.

Smith

Thomas

Snoswell

, et al. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020; 26: 309–313.

51.

Kennedy

Arey

Hopkins

, et al. Dermatologist perceptions of teledermatology implementation and future use after COVID-19: Demographics, barriers, and insights. JAMA Dermatol 2021; 157: 595.

52.

Dovigi

Kwok

EYL

English

. A framework-driven systematic review of the barriers and facilitators to teledermatology implementation. Curr Derm Rep 2020; 9: 353–361.

53.

Tensen

van der Heijden

Jaspers

MWM

, et al. Two decades of teledermatology: Current Status and integration in national healthcare systems. Curr Derm Rep 2016; 5: 96–104.

54.

Finnane

Dallest

Janda

, et al. Teledermatology for the diagnosis and management of skin cancer: A systematic review. JAMA Dermatol 2017; 153: 319–327.

55.

Coates

Kvedar

Granstein

. Teledermatology: From historical perspective to emerging techniques of the modern era: Part II: Emerging technologies in teledermatology, limitations and future directions. J Am Acad Dermatol 2015; 72: 577–586.

56.

Kownacki

. Skin diseases in primary care: What should GPs be doing? Br J Gen Pract 2014; 64: 380–381.

57.

Schofield

Grindlay

Williams

. Skin conditions in the UK: a health care needs assessment [Internet]. Centre of Evidence Based Dermatology, University of Nottingham, http://uhra.herts.ac.uk/handle/2299/5622 (2009, accessed April 2022).

58.

Lowell

Froelich

Federman

, et al. Dermatology in primary care: Prevalence and patient disposition. J Am Acad Dermatol 2001; 45: 250–255.

59.

Bae

Hartman

Joyce

, et al. Comparing dermatology referral patterns and diagnostic accuracy between nonphysician providers, physician trainees, and attending physicians. J Am Acad Dermatol 2016; 75: 226–227.

60.

Ojeda

Graells

. Effectiveness of primary care physicians and dermatologists in the diagnosis of skin cancer: A comparative study in the same geographic area. Actas Dermosifiliogr 2011; 102: 48–52.

61.

Chen

Bravata

Weil

, et al. A comparison of dermatologists’ and primary care physicians’ accuracy in diagnosing melanoma [Internet]. Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. Centre for Reviews and Dissemination (UK), https://www.ncbi.nlm.nih.gov/books/NBK68729/ (2001, accessed 28 April 2022).

62.

Jones

Jurascheck

van Melle

, et al. Dermoscopy for melanoma detection and triage in primary care: A systematic review. BMJ Open 2019; 9: e027529.