Impact of Dermoscopy Training on Diagnostic Accuracy,and Its Association With Biopsy and Referral Patterns Among Primary Care Providers: A Retrospective and Prospective Educational Intervention Study

Abstract

Background:

Proper diagnosis in primary care is crucial due to the large number of skin cancer diagnoses each year and its associated growing economic burden. Understanding how primary care providers can be best trained in dermoscopy is instrumental in helping primary care providers differentiate benign and malignant cutaneous lesions so that appropriate action can be taken (eg, biopsy/referral to dermatology or reassurance).

Objective:

To assess the success of concise dermoscopy training among primary care providers.

Design:

A cohort study was conducted in primary care at Mayo Clinic, Rochester, including primary care physicians, nurse practitioners, and physician assistants in internal medicine and family medicine. Diagnostic accuracy was measured on standardized testing through image-based recognition and in clinical practice through observation of referral and biopsy patterns both before and after dermoscopy education, focused on the TADA plus method.

Results:

Forty-three primary care providers completed the image-based recognition tests, with significant improvement in mean score measured after dermoscopy training workshop (20%). Among the thirteen primary care providers who continued to use dermoscopy in clinical practice, a significant improvement (31%) in mean clinical diagnostic accuracy was observed comparing 1 year of practice data before versus after dermoscopy training.

Conclusion:

Improvement in diagnostic accuracy with utilization of the TADA plus method translates into clinical practice. Therefore, training primary care providers in dermoscopy may improve the dermatologic care patients receive in primary care, especially in rural or medically underserved areas where access to dermatology is limited.

Keywords

primary care Triage Amalgamated Dermoscopic Algorithm TADA dermoscopy skin cancer

Introduction

In the United States, skin cancer is the most common type of cancer, making accurate skin lesion assessment crucial.^1-3 The economic burden of this disease is large and increasing, further emphasizing the importance of early detection in primary care.¹ Experts recommend that all primary care providers (PCPs) be proficient using dermoscopy,⁴ which can reduce unnecessary biopsies and referrals of benign lesions,⁵ but only 8% to 9% of primary care physicians utilize dermoscopy, with lack of training being a frequently cited barrier, while little is know about advanced practice providers specifically.^6,7 Existing literature evaluating diagnostic accuracy for skin cancer amongst both dermatology and primary care providers relies heavily on the number needed to biopsy (NNB), which has numerous limitations.⁸ While attending a single dermoscopy workshop focused on the Triage Amalgamated Dermoscopic Algorithm (TADA; Figure 1) is known to improve diagnostic accuracy on image recognition tests and decrease the NNB,^9-14 existing evidence does not show if and how this translates to actual primary care clinician behavior/patient care outside of a NNB measure alone. TADA is a 3-step triage process that can be used to aid a clinician in discerning between benign and malignant lesions. The triage tool is outlined in detail in Figure 1.

Figure 1.

Triage Amalgamated Dermoscopic Algorithm (TADA).

Dermoscopy training has the ability to lower biopsies/referrals long-term and improve overall diagnostic accuracy for PCPs and Dermatology providers. For the PCP, dermoscopy utilization can assist in better triaging lesions to determine if they are benign versus malignant so that the most appropriate next step can be taken. In rural or medically underserved areas, increased PCP skin lesion knowledge and skill is especially important with literature showing that increased melanoma mortality is directly correlated with the distance traveled to the diagnosing clinician. Given the current gap in the literature and importance of dermoscopy, we sought to evaluate the potential associations between test-based diagnostic accuracy and clinical diagnostic accuracy, by training a cohort of PCPs to use dermoscopy and comparing changes of their image recognition abilities to referral and biopsy patterns (both before and after an educational intervention).

Methods

All PCPs from the Divisions of Family Medicine and Community Internal Medicine at Mayo Clinic in Rochester, MN were invited via email to participate in an educational study on dermoscopy. At this institution, a PCP is defined as a physician (MD/DO) or advanced practice provider (Nurse Practitioner or NP and Physician Assistant or PA). Enrollees (n = 55) completed a pre-test of 30 dermoscopy images before undergoing a 2-h asynchronous course, in which 2 experienced dermoscopists (HTM & DLS) taught the TADA Plus method where the TADA method was taught along with additional training on common benign lesions, AK/SCC spectrum, and benign nevi versus melanoma lesions. Participants had a period of 2 weeks to complete the online training that was delivered via recorded presentations. An image recognition post-test was administered after enrollees completed the online education (n = 43), and participants were taught how to physically operate a dermatoscope during a 30-min hands-on workshop (n = 23). Of note, the 30-question pre- and post-test was developed by the research team using expert or dermatopathology verified images. Only PCPs who completed both didactic and technical training activities (n = 23) were included for further study (Figure 2).

Figure 2.

Study design.

A 1-year retrospective review of participants’ billing data and visit notes was performed to calculate the pre-training diagnostic accuracy of each individual. Diagnoses unlikely to be aided by dermoscopy were excluded (eg, rashes, warts, cysts). Lesions were excluded where PCPs’ assessments were unclear or diagnoses could not be confirmed (if no biopsy or dermatology referral was completed). Each PCP’s pre-training diagnostic accuracy was calculated as the percent of agreement between their clinical impressions and those of consulting dermatologists (or pathology results, if biopsied).

After TADA plus training, participants’ dermoscopy-related practices were prospectively observed for 1 year to evaluate potential changes to PCP biopsy and referral patterns. Skin lesions examined by PCPs were photographed (including with dermoscopy), reported as benign or malignant, and managed accordingly. Post-training PCP diagnostic accuracy was calculated as the percent agreement between their clinical impressions and biopsy results or a dermatologist’s impression based on image review (DLS or JCS). Only PCPs who recorded at least 5 lesions (n = 13) were included in further analyses.

Changes in participants’ post-training standardized test scores and clinical diagnostic accuracy were compared using a paired t-test. Differences between each PCP’s clinical diagnostic accuracy before and after training were compared using Fisher exact tests, and compared to dermatologist impressions using a paired t-test. Analyses were completed using SAS (SAS Institute Inc., version 9.4). The study was approved by the Mayo Clinic Institutional Review Board on October 10, 2022. See Figure 1 for flow-chart of methodology

Results

Forty-three PCPs completed both the pre- and post-tests, including 17 physicians (46%), 17 NPs (38%), and 9 PAs (16%). Most participants had over 3 years of primary care experience (87%), and 98% had little or no prior dermoscopy training. Total mean scores on written tests improved significantly after the training workshop (20.0%), as did scores for distinguishing benign from malignant lesions (19.4%). Significant improvement was noted for all but 2 diagnoses: actinic keratosis and benign nevi (Table 1).

Table 1.

Pre/Post Results for Those Who Completed Both Tests.

	Pre (N = 43)	Post (N = 43)	P-value
Overall score (out of 30)			<.001^a
Mean (SD)	12.6 (3.54)	17.4 (3.83)
Median	12.0	18.0
Range	7.0, 20.0	9.0, 24.0
Benign score (out of 17)			<.001^a
Mean (SD)	7.6 (2.79)	10.4 (2.86)
Median	7.0	11.0
Range	3.0, 13.0	2.0, 15.0
Malignant score (out of 13)			<.001^a
Mean (SD)	5.0 (1.38)	7.0 (1.77)
Median	5.0	7.0
Range	2.0, 7.0	4.0, 11.0
Percent correct by question type (Mean (SD))
Malignant melanoma	55.3% (15.0)	73.5% (17.3)	<.001^a
Malignant—basal cell carcinoma	16.9% (20.2)	32.6% (24.7)	<.001^a
Malignant—squamous cell carcinoma	38.4% (34.2)	64.0% (35.1)	0.001^a
Pre-malignant—actinic keratosis	39.5% (38.7)	34.9% (35.4)	0.499^a
Benign—dermatofibroma	40.3% (36.8)	70.5% (30.2)	<.001^a
Benign–typical melanocytic nevus	64.5% (33.3)	47.1% (30.0)	0.001^a
Benign—angioma	51.2% (25.6)	72.7% (17.1)	<.001^a
Benign—seborrheic keratosis	29.5% (25.2)	58.5% (19.0)	<.001^a

Paired t-test P-value.

Thirteen PCPs (1 physician, 10 NPs, and 2 PAs), submitted 5 or more lesions prospectively and were included in final analyses. PCPs recorded a collective total of 249 skin lesions, which were ultimately diagnosed clinically by a dermatologist (84.7%) or histologically by a dermatopathologist (15.3%). The low count of dermatopathology verified lesions is expected in primary care where the majority of lesions examined are benign and do not require biopsy. The most common diagnoses were benign nevus (45.8%), seborrheic keratosis (24.5%), and melanoma (5.6%). A significant improvement (31%) in mean clinical diagnostic accuracy was observed comparing 1 year of practice data before versus after dermoscopy training (Table 2). Both retrospective and prospective datasets were analyzed to determine the diagnostic concordance between PCPs and dermatologists. When PCPs categorized a lesion as benign or malignant, their impressions were compared to those of dermatologists (or histology reports, if biopsied). Five participants showed statistically significant improvement in classifying lesions post-training; eight showed changes that did not reach statistical significance (five improved, three decreased; Table 2). For all 13 clinicians, their concordance with the dermatology team was tabulated to explore potential trends (Table 3).

Table 2.

Percent Correct for Each Provider in the Retrospective and Prospective Samples.

Provider	Retrospective sample		Prospective sample		P-value
Provider	N samples	Diagnostic accuracy^a (%)	N samples	Diagnostic accuracy^a (%)	P-value
1	17	41.2	14	85.7	.025^b
2	15	41.2	30	76.7	.026^b
3	5	40.0	15	73.3	.290^b
4	8	50.0	14	85.7	.137^b
5	25	68.0	88	65.9	.999^b
6	20	45.0	6	83.3	.170^b
7	16	31.3	6	16.7	.634^b
8	11	45.5	11	81.8	.183^b
9	14	21.4	9	77.8	.013^b
10	25	24.0	12	83.3	.001^b
11	24	45.8	11	36.4	.721^b
12	10	10.0	27	81.5	<.001^b
13	10	30.0	6	66.7	.302^b
All PCPs	13	39.4	13	70.4	.003^c

Diagnositic accuracy = agreement by dermatology.

Fisher’s exact test P-value.

Paired t-test P-value.

Table 3.

Further Analysis of Retrospective and Prospective Clinical Performance.

Those in red columns signify disagreement between PCP and dermatologist/pathology.

- Green highlighted columns represent correctly identified lesions.

- Red highlighted columns represent incorrectly identified lesions.

Discussion

To our knowledge, no studies have been conducted to analyze the impact of TADA with PCPs beyond standardized testing and one study evaluating NNB. The purpose of this study was to implement concise dermoscopy training focused on the TADA plus method and determine if subsequent PCP dermoscopy use translated into meaningful diagnostic improvement.

Our study demonstrated that after TADA training, PCPs significantly improved their ability to assess dermoscopy images of benign and malignant lesions, outside of benign melanocytic nevi and actinic keratoses where they fared slightly worse. With the TADA algorithm centered around the following 3 common benign lesions: angioma, dermatofibroma, and seborrheic keratosis, it is unsurprising that PCPs performed well for these 3 lesion types specifically with significant improvements measured on testing. There was a significant increase in diagnostic accuracy for all 3 types of malignant lesions, with the best performance on melanomas. We hypothesize that PCPs did not perform well with actinic keratoses due to the fact that no actinic keratoses case studies were utilized during training. Rather, only the basics of actinic keratosis lesions were taught. PCPs may have low risk tolerance when evaluating melanocytic lesions, erring in favor of biopsies/referrals to avoid missing melanomas if even slight ambiguity was present.

When measuring clinical diagnostic accuracy for the PCP group (N = 13) 1-year post training, a significant improvement was measured overall, demonstrating that the TADA plus method can effectively assist the clinician in evaluating skin lesions in the primary care setting. While only 5 PCPs had statistically significant improvement in diagnostic accuracy in clinical practice, improvement was observed for 10/13 (76.9%) of the clinicians. Further analysis shows that the lack of improvement observed amongst the 3 PCPs with decline in diagnostic accuracy was due largely to an observed behavior of referring/biopsying benign lesions, not missed malignant lesions. Of note, two of the PCPs did not see any malignant lesions and one PCP did see seventeen malignant lesions, correctly diagnosing 70.6% of them, over the course of 1 year following training. A closely mirrored scenario is observed amongst the 5 clinicians who had improvement that was not statistically significant. A similar hypothesis as above is made that the PCP was being overly cautious in the evaluation of the benign skin lesions and referring due to lack of absolute confidence in its benign quality. Increase in biopsy or referral for the first year after dermoscopy training has been documented in the literature and is expected due to the increase of information that dermoscopy provides. Therefore, it is unsurprising that this trend was observed amongst several providers. However, with continued dermoscopy use, biopsy and referral rate has been shown to improve further past the first year of utilization.¹⁶

Despite this observed cautious referring trend amongst some individuals, TADA seems to solidify the PCP’s overall confidence for benign lesions when comparing clinical diagnostic accuracy before and after dermoscopy training as a group (Table 3). One year before training, PCPs appropriately offered reassurance with 19.3% of lesions to dermatology (true negative) but referred or biopsied 58.9% of benign lesions (false positive). Conversely, 1 year after training, PCPs offered reassurance with 62.3% of benign lesions (true negative), and reduced biopsy/referral of benign lesions to 22.9%. Improvement in benign lesion recognition was the most significant improvement noted in the study, which can potentially reduce unnecessary referrals, dermatology wait time, healthcare cost, and patient anxiety. For malignant lesions, PCPs appropriately referred/biopsied 62% (23/41) of malignant lesions, with 9.2% of assessed lesions being correctly identified as malignant (true positive) and 5.6% of lesions not being appropriately referred/biopsied (false negative), compared to 20.3% and 1.5% retrospectively. One clear limitation with the retrospective data is that all false negatives could not potentially be included in the dataset because no referral or biopsy was done, thus no review of the PCP’s final diagnosis. This could explain the slightly higher false negative rate after dermoscopy training. However, the PCP diagnosis was in concordance with the dermatology for 71.5% of the lesions after dermoscopy training, which was increased from only 39.5% before training. These findings support concise dermoscopy training and utilization centered around the TADA plus method in the primary care setting by both physicians and advanced practice providers. Further research is recommended to observe the diagnostic accuracy beyond 1-year post training to identify if continued improvements are noted as previously documented in the literature.

Strengths of the study include the dermoscopy educational program evaluating both learner acquisition of knowledge and specific clinical outcomes. Limitations include a small number of participants in the clinical component of the study (N = 13) compared to initial enrollment in the pre- and post-test (N = 43). The study included only one physician in the final analysis but all primary care providers in the department were invited including physicians (MD/DO) and advanced practice providers (NP/PA). The study was only completed at one academic institution. The study categorized lesions as either benign or malignant, but it should be recognized that borderline or dysplastic lesions are common. Therefore, supplementing good dermoscopy use with a thorough history, patient symptoms, and naked eye-exam is critical. Retrospectively, the research team was unable to verify skin lesions that were not referred to dermatology or biopsied, and prospectively, skin lesions that were not consented by the patient could not be included in the study per the IRB. In other words, this limitation potentially introduces the opportunity to miss a correctly identified benign lesion retrospectively. Prospectively, we were unable to precisely conclude that there were no overlooked early melanomas, as the patients were not observed beyond 1 year, and therefore, an early melanoma that didn’t have any worrisome features at the time of evaluation could have been missed. A longer study duration of 1+ years would be needed to identify if any patient presented back to the clinic with a melanoma previously thought to be benign.¹⁷

Conclusion

Improvement in diagnostic accuracy with utilization of the TADA plus method translates into clinical practice. Training PCPs in dermoscopy may improve the dermatologic care patients receive in primary care. This is especially important in rural and medically underserved areas with limited access to dermatologists, because the stage of melanoma at time of diagnosis can be associated with distance traveled to their diagnosing clinician.^18,19 Future research could explore utilization of dermoscopy and teledermoscopy in primary care and its impact on the healthcare cost associated with treating cutaneous malignancies, access to needed dermatologic care, and patient satisfaction.

Supplemental Material

sj-docx-1-jpc-10.1177_21501319241296625 – Supplemental material for Impact of Dermoscopy Training on Diagnostic Accuracy, and Its Association With Biopsy and Referral Patterns Among Primary Care Providers: A Retrospective and Prospective Educational Intervention Study

Supplemental material, sj-docx-1-jpc-10.1177_21501319241296625 for Impact of Dermoscopy Training on Diagnostic Accuracy, and Its Association With Biopsy and Referral Patterns Among Primary Care Providers: A Retrospective and Prospective Educational Intervention Study by Hayden T. Middleton, David L. Swanson, Julio C. Sartori-Valinotti, Danielle J. O’Laughlin, Peter A. Young, Stephen P. Merry, Kelly Nelson, Karen Fischer, Renee M. Weatherly and Christopher L. Boswell in Journal of Primary Care & Community Health

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This publication was made possible by the Mayo Clinic CTSA through grant number UL1TR002377 from the National Center for Advancing Translational Sciences (NCATS), a component of the National Institutes of Health (NIH).

IRB

Our protocol was reviewed by the Mayo Clinic institutional review board and deemed exempt, non-human subjects research.

ORCID iDs

Hayden T. Middleton

Danielle J. O’Laughlin

Stephen P. Merry

Christopher L. Boswell

Supplemental Material

Supplemental material for this article is available online.

References

Guy

Machlin

Ekwueme

Yabroff

KR.

Prevalence and costs of skin cancer treatment in the US, 2002–2006 and 2007–2011. Am J Prev Med. 2015;48:183-187.

Guy

Thomas

Thompson

Watson

Massetti

Richardson

LC.

Vital signs: melanoma incidence and mortality trends and projections—United States, 1982–2030. MMWR Morb Mortal Wkly Rep. 2015;64(21):591-596.

American Cancer Society. Cancer Facts & Figures 2022. American Cancer Society; 2022.

Tran

Cyr

Verdieck

, et al Expert consensus statement on proficiency standards for dermoscopy education in primary care. J Am Board Fam Med. 2023;36(1):25-38. doi:10.3122/jabfm.2022.220143R1

Marchetti

Marghoob

AA.

Dermoscopy: not just for dermatologists. Melanoma Manag. 2015;2(1):63-73. doi:10.2217/mmt.14.32

Morris

Alfonso

Hernandez

Fernandez

MI.

Examining the factors associated with past and present dermoscopy use among family physicians. Dermatol Pract Concept. 2017;7(4):63-70 doi:10.5826/dpc.0704a13

Williams

Marghoob

Seiverling

Usatine

Tsang

Jaimes

Perspectives on dermoscopy in the primary care setting. J Am Board Fam Med. 2020;33(6):1022-1024. doi:10.3122/jabfm.2020.06.200238

Nelson

Swetter

Saboda

Chen

Curiel-Lewandrowski

Evaluation of the number-needed-to-biopsy metric for the diagnosis of cutaneous melanoma: a systematic review and meta-analysis. JAMA Dermatol. 2019;155(10):1167-1174. doi:10.1001/jamadermatol.2019.1514

Rogers

Marino

Dusza

, et al A clinical aid for detecting skin cancer: the Triage Amalgamated Dermoscopic Algorithm (TADA). J Am Board Fam Med. 2016;29(6):694-701. doi:10.3122/jabfm.2016.06.160079

10.

Seiverling

Ahrns

Stevens

, et al Dermoscopic lotus of learning: implementation and dissemination of a multimodal dermoscopy curriculum for primary care. J Med Educ Curr Dev. 2021;8:2382120521989983. doi:10.1177/2382120521989983

11.

Seiverling

Ahrns

Greene

, et al Teaching benign skin lesions as a strategy to improve the Triage Amalgamated Dermoscopic Algorithm (TADA). J Am Board Fam Med. 2019;32(1):96-102. doi:10.3122/jabfm.2019.01.180049

12.

Cyr

Craig

Ahrns

Stevens

Wight

Seiverling

Teaching skin cancer detection to medical students using a dermoscopic algorithm. PRiMER. 2021;5:6. doi:10.22454/PriMER.2021.304379

13.

Middleton

Swanson

Sartori-Valinotti

O’Laughlin

Pham

Boswell

CL.

Utility of dermoscopy training in improving diagnostic accuracy of skin lesions among physician assistant students. J Physician Assist Educ. 2023;35:9-13. doi:10.1097/JPA.0000000000000538

14.

Seiverling

Stevens

Dorr

, et al Impact of multimodal dermatoscopy training on skin biopsies by primary care providers. J Am Acad Dermatol. 2022;87(5):1119-1121. doi:10.1016/j.jaad.2022.01.027

15.

Rogers

Marino

Dusza

Bajaj

Marchetti

Marghoob

Triage Amalgamated Dermoscopic Algorithm (TADA) for skin cancer screening. Dermatol Pract Concept. 2017;7(2):39-46.

16.

Terushkin

Warycha

Levy

Kopf

Cohen

Polsky

Analysis of the benign to malignant ratio of lesions biopsied by a general dermatologist before and after the adoption of dermoscopy. Arch Dermatol. 2010;146(3):343-344. doi:10.1001/archdermatol.2010.12

17.

Matsumoto

Wack

Weinstock

, et al Five-year outcomes of a melanoma screening initiative in a large health care system. JAMA Dermatol. 2022;158(5):504-512. doi:10.1001/jamadermatol.2022.0253

18.

Glazer

Farberg

Winkelmann

Rigel

DS.

Analysis of trends in geographic distribution and density of US dermatologists. JAMA Dermatol. 2017;153(4):322-325. doi:10.1001/jamadermatol.2016.5411

19.

Stitzenberg

Thomas

Dalton

, et al Distance to diagnosing provider as a measure of access for patients with melanoma. Arch Dermatol. 2007;143(8):991-998. doi:10.1001/archderm.143.8.991

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB