Does Double Reading of Screening Breast MRI Scans Impact Recall Rates and Cancer Detection?

Abstract

French

Objective: To investigate the effect of double reads by a second radiologist on cancer detection rate (CDR), positive predictive value of recommendation for tissue diagnosis (PPV2), and the positive predictive value of biopsy performed (PPV3) for biopsy recommendations in high-risk screening breast MRIs. Methods: The policy of second reads on biopsies recommended for MRIs was prospectively implemented in October 2019. This IRB approved retrospective analysis compared consecutive high-risk screening breast MRI scans performed in a single academic institution between 06/01/2018 to 06/01/2019 (pre-intervention) with screening breast MRI scans performed between 10/31/2019 to 10/31/2020 (post-intervention). Pathology results after biopsy were recorded. Testing of association was performed using the Chi-square test. Results/Discussion: A total of 1124 screening breast MRIs in the pre-intervention and 1672 screening breast MRIs were performed in the post-intervention periods. Biopsies were recommended in 8.6% (97/1124) of pre-intervention and 5.5% (92/1672) of post-intervention MRIs (P = .0012). There was a non-significant increase in PPV2 from pre-intervention 10.3% (10/97) to post-intervention 18.4% (17/92) (P = .109) and in PPV3 from 14% (10/71) to 22.9% (17/74), respectively (P = .17). Similar cancer detection rates, 8.9/1000 (10/1124) and 10.2/1000 (17/1672) (P = .736) were diagnosed in pre-intervention and post-intervention periods, respectively. Conclusion: Double reading of screening breast MRI scans significantly reduced the number of unnecessary biopsies without significant impact in the PPVs or cancer detection rate.

Visual Abstract

Keywords

double read MRI cancer detection rate positive predictive value of recommendation for tissue diagnosis positive predictive value of biopsies performed recall rates

Introduction

Breast magnetic resonance imaging (MRI) is an essential tool in breast imaging and has been demonstrated to be the most sensitive modality for detecting breast carcinoma.¹ The combination of breast MRI and mammography has been shown to reduce breast cancer mortality by early detection of breast cancer in women at high risk of breast cancer.^2,3 The 2013 5th edition of the American College of Radiology’s Breast Imaging Reporting and Data System (BI-RADS) Atlas provides performance benchmarks for screening breast MR imaging.⁴ Screening benchmarks include cancer detection rate (CDR), positive predictive value of recommendation for tissue diagnosis (PPV2), positive predictive value for biopsies performed (PPV3), recall rates, and multiple other metrics. Screening breast MRI performance in community settings compared with these benchmarks have shown favorable outcomes.⁵

There is renewed debate about the benefits and cost of a single read vs double read of breast imaging studies.⁶ Previous studies have shown that double reading of mammograms by a second radiologist reduced recall rate and increased cancer detection.⁷ Other studies have shown an increased CDR with an increased recall rate and false positive rate.⁸ So far, no study has evaluated whether the double reading of MRI examinations by a second radiologist has the same effect. Therefore, the objective of our study was to determine whether double reading of MRI scans in our institution had an impact on the performance measures outlined by the BI-RADS 5th edition.

Methods

Study Design and Patients

With institutional research ethics board approval and need for consent waived, we examined all consecutive breast MRI studies performed at our institution from June 1, 2018 to June 1, 2019 and from October 31, 2019 to October 31, 2020. These dates were chosen as the policy of having second reads for breast MRI studies at our institution began in October 2019. With the implementation of this policy, a second read by a separate radiologist was requested only when a screening breast MRI case was felt to warrant a biopsy (e.g., BI-RADS 4 and 5 cases). This policy was instituted as a quality assurance project to improve PPV of biopsy recommendations to be more in line with published ACR benchmarks. With the implementation of this policy, a second read by a separate radiologist was requested only when a screening breast MRI case was felt to warrant a biopsy (e.g., BI-RADS 4 and 5 cases). Disagreements between the two readers were resolved by group consensus and further management was determined by the agreed BI-RADS category.

In Ontario, all patients undergoing a screening breast MRI study were women ages 30–69 who have a 25% or greater lifetime risk of breast cancer.⁹ Women with non-screening indications, which included problem solving MRI studies and MRI studies that assessed for implant rupture, were excluded.

MRI Technique and Definitions

Breast MRI were performed according to established institution shortened MRI protocol:¹⁰ T1 nonfat-suppressed coronal ((GRAPPA)300, .6 × .6 × .6 mm isotropic, 3:03 min, 6.4/2.57), T2 fat suppressed (.5 × .5 × 3.0 mm, 3:07 min, 12 250/132.0), dynamic T1 fat suppressed (1 pre, 3 post) (3D VIBE, (SPAIR)300, .97 × .97 × .97 mm isotropic, 5:36 min, 4.1/1.96) (Siemens Magnetom 3T, Erlangen, Germany).

A screening breast MRI examination was considered as positive or abnormal when additional diagnostic imaging or biopsy was recommended prior to the next routine screening examination, and these include studies that were categorized as BI-RADS 0, 3, 4, and 5. PPV2 was defined as the percentage of all screening MRI studies that recommended biopsy that resulted in a tissue diagnosis of cancer, PPV3 as the percentage of all biopsied cases that resulted in a tissue diagnosis of cancer, CDR as the number of cancers detected by MRI per 1000 studies, and the abnormal interpretation rate (AIR) as the percentage of all MRI studies that were classified as BI-RADS 0, 3, 4, and 5.

Patients with MRI studies that were designated as BI-RADS 4 and 5 were recommended to undergo biopsy. However, not all BI-RADS 4 and 5 lesions were biopsied due to a variety of reasons, such as patient factors and patient preference. If a biopsy was performed, they were performed with MRI, ultrasound, or mammographic guidance. Biopsy samples subsequently underwent pathological examination for diagnosis. While stratification of BI-RADS 4 category into 4a, 4b, and 4C is not required by the American College of Radiology (ACR), it is routinely performed at our institution to assist with the prioritization of booking MRI and ultrasound biopsies.

Data Collection

The initial database search identified 4477 MRI studies. 1681 of these MRI studies were excluded as they had non-screening indications, resulting in 2796 total screening breast MRI studies. Of these screening breast MRI studies, 1124 MRI studies were performed in the pre-intervention period from June 1, 2018 to June 1, 2019, while 1672 MRI studies were performed in the post-intervention performed from October 31, 2019 to October 31, 2020. These dates were chosen as the start of the policy of having second reads for MRI examinations in October 2019. The gap of 5 months between the pre and post intervention periods was to allow comparison of two distinct periods with a new hospital RIS established on June 1, 2019. Patient inclusion and exclusion criteria are summarized in Figure 1.

Figure 1.

Flow diagram depicting inclusion and exclusion criteria.

Statistical Analysis

Comparison between the pre-intervention and post-intervention periods was used using the chi-square test. Calculations were performed using a Chi-Square Test Calculator (Social Science Statistics, 2018).

Figure 2.

(A) 48-year-old female patient for high-risk screening breast MRI. The current study demonstrates a focal area of non-mass enhancement (NME) within the upper outer quadrant of the right breast (arrow). (B) MRI performed 12 months prior. Initially, it was thought that this area of NME had increased, and the reporting radiologist considered recommending an MRI breast biopsy. (C) However, after a second read, it was found that this area of NME was comparable to an MRI performed 18 months prior and biopsy was not recommended.

Results

A total of 1124 screening breast MRI scans were performed in the pre-intervention period, while 1672 screening breast MRI scans were performed in the post-intervention period.

The number of cases stratified by BI-RADS category, number of biopsied cases, number of diagnosed cancers, PPV2, and PPV3 values in the pre-intervention period is provided in Table 1. The same metrics in the post-intervention period are provided in Table 2. Comparison of BI-RADS category stratification between the pre-intervention and post-intervention periods is outlined in Table 3. Most notably, there was a significant decrease of 4A [5.5% (62/1124) vs 3.3% (55/1672) (P = .004) and 4B (2.5% (29/1124) vs 1.4% (24/1672)] cases from the pre-intervention and post-intervention periods.

Table 1.

Initial Assessment BI-RADS Category for Biopsies Performed During the Pre-intervention Period.

BI-RADS Category	Number of Cases (Percent of Total)	Number of Biopsied Cases	Number of Diagnosed Cancers	PPV2	PPV3
0	27 (2.4%)	5	-	-	-
3	75 (6.7%)	11	1	-	-
4A	62 (5.5%)	41	1	1.6%	2.4%
4B	29 (2.5%)	25	6	21%	24%
4C	5 (.4%)	4	2	40%	50%
5	1 (.1%)	1	1	100%	100%
4 and 5	97 (8.6%)	71	10	10.3%	14%

Table 2.

Initial Assessment BI-RADS Category for Biopsies Performed During the Post-intervention Period.

BI-RADS Category	Number of Cases (Percent of Total)	Number of Biopsied Cases	Number of Diagnosed Cancers	PPV2	PPV3
0	37 (2.2%)	13	-	-	-
3	113 (6.8%)	7	-	-	-
4A	55 (3.3%)	40	5	9%	13%
4B	24 (1.4%)	21	6	25%	29%
4C	8 (.5%)	8	3	38%	38%
5	5 (.3%)	5	3	60%	60%
4 and 5	92 (5.5%)	74	17	18.4%	22.9%

Table 3.

Screening Breast MRI BI-RADS Categories: Pre-intervention Compared with Post-intervention Periods.

BI-RADS Category	Pre-intervention (percent of total)	Post-intervention (percent of total)	P value
0	27 (2.4%)	37 (2.2%)	.743
1	399 (35.5%)	641 (38.3%)	.128
2	524 (46.6%)	786 (47%)	.839
3	75 (6.7%)	113 (6.8%)	.929
4A	62 (5.5%)	55 (3.3%)	.004
4B	29 (2.5%)	24 (1.4%)	.029
4C	5 (.4%)	8 (.5%)	.898
5	1 (.1%)	5 (.3%)	.239
6	2 (.2%)	3 (.2%)	.993
Total	1124	1672	-

Biopsies were recommended in 8.6% (97/1124) of pre-intervention and 5.5% (92/1672) of post-intervention screening breast MRI scans (P = .0012). 10/1124 and 17/1672 cancers were diagnosed in the pre-intervention and post-intervention periods, respectively, with similar cancer detection rates, 9.9/1000 and 10.2/1000, respectively (P = .736). There was an increase in PPV2 from the pre-intervention 10.3% (10/97) to post-intervention 18.4% (17/92) (P = .109) and in PPV3 from 14% (10/71) to 22.9% (17/74), respectively (P = .17). There was a decrease in abnormal interpretation rates from the pre-intervention 17.7% to post-intervention 14.5% (P = .022). These and other performance indicators are summarized in Table 4. Of note, the CDR, PPV2, and PPV3 performance indicators in the post-intervention period were closer to the 2013 ACR BI-RADS Benchmarks. No difference was noted between BI-RADS 0 [2.4% (27/1124) and 2.2% (37/1672) in the pre-intervention and post-intervention period, respectively] (P = .743) and BI-RADS 3 [6.7% (75/1124) and 6.8% (113/1672) in the pre-intervention and post-intervention period, respectively) (P = .929) rates between the two periods.

Table 4.

Summary of Performance Indicators in the Pre-intervention and Post-intervention Periods.

Performance Indicator	Pre-intervention	Post-intervention	P value	ACR BI-RADS Benchmark
Total number of patients	1124	1672	-	-
AIR (%)^a	17.7	14.5	.022	-
Total cancers detected	10	17	-	-
CDR (per 1000 examinations)	8.9	10.2	.736	20–30
Biopsies recommended	97	92	.001	-
Biopsies performed	71	74	.239	-
PPV2 (%)	10.3	18.4	.109	15
PPV3 (%)	14	22.9	.169	20–50

^aNote: The AIR included BI-RADS 0, 3, 4, and 5 studies.

Some of the BI-RADS 0 categories in our study were lesions with benign features (e.g., lymph nodes or fibroadenomas) where a second-look ultrasound was recommended for verification. When adhering to the standardized ACR lexicon, these cases would have been incorrectly characterized as BI-RADS 0 and would have been more correctly categorized as BI-RADS 4A. After review of all BI-RADS 0 lesions in the two periods, it was noted that this error occurred in small numbers (11.1% (3/27) and 16.2% (6/37) in the pre-intervention and post-intervention periods, respectively).

Discussion

In this retrospective, population-based, cohort study, we evaluated the effect of double read MRIs and their effect on multiple performance metrics. Our data shows that a second reader reduced the overall number of biopsies without affecting the other performance indicators, including PPV2, PPV3, and cancer detection rates. Although the first few months (March 2019–June 2019) of the post-intervention period was affected by the COVID-19 pandemic and had fewer screening breast MRI scans, it had no bearing on the final outcomes of the study.

Previous studies have shown that double readings of mammography by a second reader with arbitration reduces recall rates and increase cancer detection when compared with single reading.⁷ To our knowledge, our study is the first to evaluate the effect of double readings of screening MRI examinations by a second radiologist.

The process of double reading has been criticized to be time-consuming and not cost effective, with higher false positive rates.^11,12 However, our study has shown opposite results, with overall increase in PPV2 and PPV3. Fewer biopsies were recommended, and fewer biopsies were performed, reducing the overall MRI time needed to perform the image-guided biopsy. This can potentially decrease overall healthcare costs. Furthermore, second reads were only requested in cases that were initially interpreted as BI-RADS 4A, 4B, 4C, or 5, which constituted a small percentage of all MRIs performed Figure 2. Double reads were also easily requested and feasible with the chat function on our electronic medical record system (Epic Systems Corporation, Verona, WI, USA). This chat function allowed swift communication and sharing of cases between radiologists, taking an average of 5 minutes per case. Nevertheless, more research needs to be performed to determine the true cost-effectiveness of double read MRI studies and their long-term health outcomes.

Our study shows that double reading of MRI scans particularly affected BI-RADS 4A and 4B categories, with decrease in the overall number of cases in the two BI-RADS categories. The overall percentage of BI-RADS 4A cases in the pre-intervention period was 5.5% (62/1124) and 3.3% (55/1672) in the post-intervention period (P = .004). The overall percentage of BI-RADS 4B cases in the pre-intervention period was 2.5% (29/1124) and 1.4% (24/1672) in the post-intervention period (P = .029). The overall decrease in BI-RADS 4A and 4B cases reduced the overall number of cases referred for biopsies and improved PPV 2 and PPV3. It can be postulated that MRI cases with findings of lower suspicion or areas with questionable findings (BI-RADS 4A and 4B) may have been more challenging to read and interpret, and therefore benefited more from double reads.

Our study has limitations. The double reads performed by a second radiologist were unblinded, which may bias the second reader to agree with the initial reader’s recommendation for biopsy. However, despite this risk of bias, our data shows that overall biopsies of cases had decreased after the double read policy. Another limitation of our study was the incorrect usage of BI-RADS 0 category in cases where a second-look ultrasound was recommended for verification, which would have been more correctly characterized as BI-RADS 4A lesions. However, this error was only seen in a few studies in the pre-intervention and post-intervention periods (11.1% (3/27) and 16.2% (6/37) in the pre-intervention and post-intervention periods, respectively). Finally, in our study, we restricted double reads to MRI studies that were initially interpreted as BI-RADS 4 or 5. It is therefore possible that some cancers were not detected and did not have a second read as the initial reader felt the study did not warrant a biopsy. We did not assess if double reading all cases would have resulted in a higher cancer detection rate or impacted the biopsy rate. A previous study by Marshall et al. found that review and discussion of BI-RADS 3 cases with at least two other radiologists reduced the overall number of BI-RADS 3 assessments.¹³ Although double read of BI-RADS 3 cases were not reviewed in our study, this policy could be considered in the future. Double reading can also be used in conjunction with artificial intelligence (AI) contribution in the future.

In conclusion, the use of a second read in screening breast MRI scans significantly reduced the number of unnecessary biopsies without significant impact in the PPVs or cancer detection rate.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jean Seely

References

DeMartini

Lehman

. A review of current evidence-based clinical applications for breast magnetic resonance imaging. Top Magn Reson Imag [Internet]. 2008;19(3):143–150. Available from: http://journals.lww.com/00002142-200806000-00001

Bae

Sung

Han

, et al. Survival outcomes of screening with breast MRI in high-risk women. J Clin Oncol [Internet]. 2017;35(15_suppl l):1508. Available from: http://ascopubs.org/doi/10.1200/JCO.2017.35.15_suppl.1508

Daly

Pal

Berry

, et al. Genetic/familial high-risk assessment: breast, ovarian, and pancreatic, version 2.2021, NCCN clinical practice guidelines in oncology. J Natl Compr Cancer Netw [Internet]. 2021;19(1):77–102. Available from: https://jnccn.org/view/journals/jnccn/19/1/article-p77.xml

D’Orsi

Sickles

Mendelson

Morris

. ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. American College of Radiology. Reston, VA; 2013.

Lee

Ichikawa

Valencia

, et al. Performance benchmarks for screening breast MR imaging in community practice. Radiology [Internet]. 2017;285(1):44–52. Available from: http://pubs.rsna.org/doi/10.1148/radiol.2017162033

Euler-Chelpin

Lillholm

Napolitano

Vejborg

Nielsen

Lynge

. Screening mammography: Benefit of double reading by breast density. Breast Cancer Res Treat [Internet]. 2018;171(3):767–776. Available from: http://link.springer.com/10.1007/s10549-018-4864-1

Taylor-Phillips

Jenkinson

Stinton

Wallis

Dunn

Clarke

. Double reading in breast cancer screening: Cohort evaluation in the CO-OPS trial. Radiology. 2018;287(3):749–757.

Coolen

AMP

Voogd

Strobbe

Louwman

MWJ

Tjan-Heijnen

VCG

Duijm

LEM

. Impact of the second reader on screening outcome at blinded double reading of digital screening mammograms. Br J Cancer [Internet]. 2018;119(4):503–507. Available from: http://www.nature.com/articles/s41416-018-0195-6

Chiarelli

Blackmore

Muradali

, et al. Performance measures of magnetic resonance imaging plus mammography in the high risk ontario breast screening program. JNCI J Natl Cancer Inst [Internet]. 2020;112(2):136–144. Available from: https://academic.oup.com/jnci/article/112/2/136/5522370

10.

Lukaszewicz

Lamb

Healey

Alie

Seely

. Abbreviated high-risk screening breast MRI protocol: Impact on cancer detection and biopsy rates. In: RSNA Abstract Presentation (RC215-04). Chicago, US; 2018.

11.

Posso

Puig

Quintana

Solà-Roca

Bonfill

. Double versus single reading of mammograms in a breast cancer screening programme: A cost-consequence analysis. Eur Radiol [Internet]. 2016;26(9):3262–3271. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26747264

12.

Posso

Carles

Rué

Puig

Bonfill

. Cost-effectiveness of double reading versus single reading of mammograms in a breast cancer screening programme. Consolaro

MEL

, ed. PLoS One [Internet]. 2016;11(7):e0159806. Available from: https://dx.plos.org/10.1371/journal.pone.0159806

13.

Posso

Puig

Carles

Rué

Canelo-Aybar

Bonfill

. Effectiveness and cost-effectiveness of double reading in digital mammography screening: A systematic review and meta-analysis. Eur J Radiol. 2017;96(April):40–49.