Sage Journals: Discover world-class research

Abstract

Objectives

To determine the agreement between artificial intelligence software (AI) and radiographers in assessing breast positioning criteria for mammograms from standard digital mammography and digital breast tomosynthesis.

Methods

Assessment of breast positioning was performed by AI and by four radiographers in pairs of two on 156 examinations of women screened in Bergen, April to September 2019, as part of BreastScreen Norway. Ten criteria were used; three for craniocaudal and seven for mediolateral-oblique view. The criteria evaluated the appearance of the nipple, breast rotation, pectoral muscle, inframammary fold and pectoral nipple line. Intraclass correlation and Cohen’s kappa coefficient (κ) were used to investigate the correlation and agreement between the radiographer’s assessments and AI.

Results

The intraclass correlation for the pectoral nipple line between the radiographers and AI was >0.92. A substantial to almost perfect agreement (κ > 0.69) was observed between the radiographers and AI on the nipple in profile criterion. We observed a slight to moderate agreement for the other criteria (κ = 0.06–0.52) and generally a higher agreement between the two pairs of radiographers (mean κ = 0.70) than between the radiographers and AI (mean κ = 0.41).

Conclusions

AI has great potential in evaluating breast position criteria in mammography by reducing subjectivity. However, varying agreement between radiographers and AI was observed. Standardized and evidence-based criteria for definitions, understandings and assessment methods are needed to reach optimal image quality in mammography.

Keywords

Artificial intelligence breast neoplasm breast screening mammography radiography

Introduction

Consistent production of high-quality images is important in mammographic screening to achieve optimal visualisation of the breast tissue and detection of abnormalities associated with breast cancer. Image quality has shown to influence the rate of recall and screen-detected cancer, and consequently the sensitivity and specificity of screening.^1,2 Several factors affect image quality, and breast positioning is one of the main contributors.

Image quality assessment can be performed directly at the screening examination or retrospectively, for quality assurance or for regular monitoring of image quality. The assessment includes evaluation of the positioning of the breast, how the breast is projected on the mammogram, and whether there is any movement blur, noise or other artefact in the image.^3,4 Different systems for image quality assessment are available; either checklists with image quality statements, or classification systems, where several image quality statements are assessed and the images are classified based on the overall score.^3–6 Commonly used systems include the ‘perfect, good, moderate, inadequate’ (PGMI) system and the ‘excellent, adequate, repeat’ (EAR) system.^7,8 These two systems were developed for screen-film mammography, for which image quality was not possible to assess immediately at the screening examination.

BreastScreen Norway has used a modified version of the PGMI system for image quality assessment since the program started in 1996. The modification has been revised over the years, most recently in 2011.⁹ However, PGMI classification is a time-consuming task, and only a sample of each radiographer’s images are reviewed and classified. Furthermore, performing image quality classification is subjective as the assessment differs between assessors, both for PGMI and for other classification systems, raising questions about the reliability and validity of these systems.^10–13 In 2017, UK guidelines for mammographic screening stated that PGMI and EAR are no longer acceptable methods to assess image quality.⁵ However, high-quality images are still a prerequisite in order to achieve a high sensitivity in diagnostic as well as screening mammography. An image quality assessment system that is reliable, valid and objective is therefore needed. As of today, most studies on image quality and positioning are based on screen-film or digital mammography (DM), and fewer on digital breast tomosynthesis (DBT).

In recent years, automated systems for image quality assessment using artificial intelligence (AI), have been developed.^14–17 An AI system for image quality assessment would eliminate subjectivity and reduce time spent assessing image quality by the radiographers. Furthermore, having immediate image quality assessment at the screening before the woman leaves the screening unit could reduce recalls due to inadequate image quality. However, the development of automated systems relies on human observers to provide the reference standard for choice of image quality criteria and thresholds which will affect the results of the assessment.¹⁸ Research is needed to investigate whether the AI systems for image quality assessment provide a valid and reliable measure of the criteria. In this study, we aimed to determine the agreement between an AI system and radiographers in assessing breast positioning criteria for DM and DBT.

Methods

We assessed breast positioning criteria from women attending the screening unit in Bergen, as part of BreastScreen Norway, which is a nationwide, population-based breast cancer screening program offering women aged 50–69 biennial two-view mammographic screening.¹⁹ The Cancer Registry of Norway administers the program, while the screening examinations are performed at 26 stationary and four mobile screening units, organised under 17 breast centres across the country. Data collection as a part of BreastScreen Norway is regulated by the Cancer Registry Regulation and the Norwegian Health Registry Act §8, and data are stored at the Cancer Registry of Norway. The Regional Committee for Medical and Health Research Ethics (reference 2015/424) approved this study.

Study population

Mammograms from a random sample of 180 DM and 180 DBT screening examinations performed during the period from April to September 2019 were identified from the databases at the Cancer Registry of Norway and constituted the study population. All examinations were performed using GE Senographe Pristina 3 D Breast Tomosynthesis™ and consisted of two-view (craniocaudal, CC; and mediolateral oblique, MLO) screening examinations performed by 27 radiographers. Synthetic mammograms were used for assessing the breast positioning criteria of DBT examinations.

Assessment of breast positioning criteria

Assessments of breast positioning were performed manually and by the AI system. The manual assessment was performed over the course of two days on a weekend in October 2019 on two ‘picture archiving and communication system’ (PACS) stations in two separate rooms with two five-megapixel screens. A dedicated study radiographer at the screening centre in Bergen identified and made the examinations available in PACS before the assessment. The assessments of the DM screening examinations were performed on one of the PACS stations and the DBT examinations on the other.

Four radiographers performed the manual breast positioning assessment in pairs of two (R1 and R2). A fifth radiographer was available for arbitration if there were any disagreements. The radiographers were between 29 and 59 years old and had at least 3 years’ experience in mammographic screening. A pilot set of ten images was assessed in plenary before the assessment started to establish a common level of understanding of the breast positioning criteria and assessment methods. None of the radiographers participating in this study were involved in the development of the AI system. The pairs used one workday on each PACS station and recorded the results in a prepared spreadsheet. We aimed to assess as many images as possible during the available time slot and included all images except those of women with breast implants. We reached a total of 156 examinations, 79 (50.6%) performed using DM and 77 (49.4%) performed using DBT (Figure 1).

Figure 1.

Flow chart of the study population.

The AI assessment was performed on all examinations included in the study population. Automated image processing software obtaining image information from the Digital Imaging and Communications in Medicine (DICOM) header and image pixel data (Volpara®Density™, Volpara Health Technologies Ltd., Wellington, NZ; algorithm version 1.5.5.1) was used.

Breast positioning criteria

Breast positioning was assessed using ten criteria; three for CC and seven for MLO view (Table 1). The breast positioning criteria were selected from the PGMI used in BreastScreen Norway⁹ and criteria available from the AI system. The criteria included nipple in profile (both views), pectoral nipple line (both views), rotation of the breast (CC view), angle of pectoral muscle (MLO view), fold in pectoral muscle (MLO view), length of pectoral muscle to posterior nipple line (MLO view), shape of pectoral muscle (MLO view) and inframammary fold visibility (MLO view).

Table 1.

Breast positioning criteria, assessment methods for manual assessment of breast positioning and associated measurements.

	Breast positioning criteria	Assessment method	Measurement
CC view	Nipple in profile	Visually	YesNo
	Pectoral nipple line	Distance measurement tool in PACS	mm
	Rotation of the breast	Goniometer	Nipple is central (no rotation of breast) Nipple is rotated 5-10°, lateral or medialNipple is rotated >10°, lateral or medial
MLO view	Nipple in profile	Visually	YesNo
	Pectoral nipple line	Distance measurement tool in PACS	mm
	Angle of pectoral muscle	Goniometer	≤20°>20°
	Fold in pectoral muscle	Visually	YesNo
	Length of pectoral muscle to posterior nipple line	Distance measurement tool in PACS	Sufficient: Pectoral muscle reaches 1 cm or more below pectoral nipple lineInsufficient: Pectoral muscle does not reach 1 cm below pectoral nipple line
	Shape of pectoral muscle	Visually	StraightConcaveConvex
	Inframammary fold visibility	Visually	Open and free of skin foldsOpen with skin foldNot included

PACS: picture archiving and communication system; MLO: mediolateral oblique.

The pairs of radiographers reviewed the images and discussed the criteria to come to an agreement. A distance measurement tool in PACS was used to measure the pectoral nipple line, and length of the pectoral muscle to posterior nipple line. The pectoral nipple line in CC view was defined as the distance from the nipple to the image edge, independent of the pectoral muscle. For MLO view, the pectoral nipple line was defined as the distance from the nipple perpendicular to the anterior margin of the pectoral muscle (or the posterior image edge). An approximation of the pectoral line was drawn as a straight line, from the anterior to the superior end of the muscle in the image. The pectoral line and the corner of a piece of standard A4 paper were used to simplify drawing the perpendicular pectoral nipple line for the MLO view (Supplementary file 1). Measurement of the length of the pectoral muscle to the posterior nipple line used the perpendicular line created for the pectoral nipple line, i.e. whether the pectoral muscle reached 1 cm or more below this line. A goniometer was used to assess the rotation of the breast (CC) and the angle of the pectoral muscle (MLO). Rotation was measured as degrees between the midline of the breast at the posterior image edge and the position of the nipple, while the angle of the pectoral muscle was measured using the pectoral line and the posterior image edge.

Statistical analyses

Mean value and standard deviation (SD) were used to present the continuous variable pectoral nipple line and intraclass correlation (ICC) was used to investigate the correlation between R1, R2 and AI. ICC estimates and their 95% confidence interval (95% CI) were calculated based on a mean-rating absolute agreement and two-way random effects model. Correlation was interpreted according to the following distribution: <0.49, poor correlation; 0.50–0.74, moderate correlation; 0.75–0.89, good correlation; 0.90–1.0, excellent correlation.²⁰ All other variables were categorical and presented as percentages, and counted for each criterion by R1, R2 and AI. Further, they were presented as percentages and counted as agreement or disagreement between R1, R2 and AI. Cohen's kappa coefficient (κ) was used to quantify the agreement between R1, R2 and AI, and all groups together. The degree of agreement was determined according to the following distribution: <0, poor agreement; 0.0–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; 0.81–1.0 almost perfect agreement.²¹ Characteristics of the women screened, including mean values and 95% CI for age at screening (years), breast volume (cm³), fibroglandular volume (cm³), volumetric breast density (%), compression force (Newton), compression pressure (kilopascal), and compressed breast thickness (mm) are shown in Supplementary file 2. We found no statistically significant differences between the results for DM versus DBT, so all results are presented for DM and DBT combined. Results for DM and DBT, separately, are given in Supplementary files 3–6. Statistical analyses were performed using Stata version 16 (StataCorp, TX, USA).

Results

Pectoral nipple line (both views)

Mean length of the pectoral nipple line for CC view was 9.9 mm (SD = 2.4) for R1, 10.0 mm (SD = 2.4) for R2 and 10.8 mm (SD = 2.7) for AI (Table 2). For MLO view, mean length of the pectoral nipple line was 10.2 mm (SD = 2.6) for R1, 10.4 mm (SD = 2.6) for R2 and 11.4 mm (SD = 2.8) for AI (Table 2). ICC for the pectoral nipple line between R1 and AI and between R2 and AI was >0.92 for both views.

Table 2.

Mean value with standard deviation (SD) of the pectoral nipple line for CC and MLO view, by R1, R2 and AI and the intraclass correlation (ICC) with 95% confidence interval (95% CI) between R1, R2 and AI for CC and MLO view.

	Pectoral nipple line
	CC	MLO
	Mean value, mm (SD)
R1	9.9 (2.4)	10.2 (2.6)
R2	10.0 (2.4)	10.4 (2.6)
AI	10.8 (2.7)	11.4 (2.8)

	ICC (95% CI)
R1 and R2	0.98 (0.97–0.98)	0.99 (0.98–0.99)
R1 and AI	0.96 (0.31–0.99)	0.92 (0.38–0.97)
R2 and AI	0.95 (0.75–0.98)	0.94 (0.53–0.98)
R1, R2 and AI	0.97 (0.92–0.99)	0.96 (0.87–0.98)

CC: craniocaudal; ICC: intraclass correlation; AI: artificial intelligence; CI: confidence interval; MLO: mediolateral oblique.

Nipple in profile (both views) and rotation of the breast (CC view)

The percentage of images with “nipple in profile” (yes) varied from 75.6% (118/156) to 83.3% (130/156) for CC and from 75.6% (118/156) to 84.6% (132/156) for MLO between R1, R2 and AI (Table 3). Substantial agreement was observed for R1 and AI, and R2 and AI for CC view (κ = 0.77 and 0.69, respectively). For MLO, the agreement was almost perfect for R1 and AI (κ = 0.83), and substantial for R2 and AI (κ = 0.72). The percentage of CC images with “central position of the nipple” (no rotation of the breast) varied between 35.9% (56/156) and 44.9% (70/156), while “nipple with a rotation” of 5°–10° varied between 19.2% (30/156) and 31.4% (49/156), and >10° varied between 24.4% (38/156) and 40.4% (63/156) (Table 3). Moderate agreement was observed between R1 and AI (κ = 0.52), and between R2 and AI (κ = 0.49).

Table 3.

Percentage (%) and proportion (n = out of 156 examinations) of nipple in profile for CC and MLO view, and rotation of the breast for CC view, by R1, R2 and AI, and the agreement, disagreement and Cohen’s Kappa value between R1, R2 and AI and in total.

	Nipple in profile – CC view			Nipple in profile – MLO view			Rotation of the breast (in degree)
	Yes	No		Yes	No		Central nipple	Nipple rotated 5–10°	Nipple rotated >10°
	% (n=)	% (n=)		% (n=)	% (n=)		% (n=)	% (n=)	% (n=)
R1	83.3% (130)	16.7% (26)		81.4% (127)	18.6% (29)		40.4% (63)	19.2% (30)	40.4% (63)
R2	82.1% (128)	17.9% (28)		84.6% (132)	15.4% (24)		35.9% (56)	31.4% (49)	32.7% (51)
AI	75.6% (118)	24.4% (38)		75.6% (118)	24.4% (38)		44.9% (70)	30.8% (48)	24.4% (38)

	Agreement	Disagreement	Kappa	Agreement	Disagreement	Kappa	Agreement	Disagreement	Kappa
R1 and R2	92.3% (144)	7.7% (12)	0.73	95.5% (149)	4.5% (7)	0.84	80.1% (125)	19.9% (31)	0.70
R1 and AI	92.3% (144)	7.7% (12)	0.77	94.2% (147)	5.8% (9)	0.83	67.9% (106)	32.1% (50)	0.52
R2 and AI	89.7% (140)	10.3% (16)	0.69	91.0% (142)	9.0% (14)	0.72	66.0% (103)	34.0% (53)	0.49
R1, R2 and AI	87.2% (136)	12.8% (20)	0.73	90.4% (141)	9.6% (15)	0.80	58.3% (91)	41.7% (65)	0.57

CC: craniocaudal; MLO: mediolateral oblique; AI: artificial intelligence.

Appearance of pectoral muscle and inframammary fold visibility (MLO view)

The percentage of images with a ≤ 20° “angle of the pectoral muscle” varied between 32.1% (50/156) and 41.7% (65/156) for R1, R2 and AI (Table 4). Fair agreement was observed between R1 and AI (κ = 0.25), and between R2 and AI (κ = 0.31). The variation in percentage of images with a “fold in the pectoral muscle” was between 12.8% (20/156) and 36.5% (57/156) between R1, R2 and AI (Table 4). The agreement between R1 and AI was fair (κ = 0.28), while it was slight for R2 and AI (κ = 0.18). The percentage of images with a “sufficient length of the pectoral muscle to posterior nipple line” varied between 62.2% (97/156) and 75.6% (118/156) between R1, R2 and AI (Table 4). Fair agreement was observed between R1 and AI (κ = 0.31), and between R2 and AI (κ = 0.27).

Table 4.

Percentage (%) and proportion (n = out of 156 examinations) of angle of pectoral muscle, fold in pectoral muscle and pectoral muscle to posterior nipple line for MLO view, by R1, R2 and AI, and the agreement, disagreement and Cohen’s Kappa value between R1, R2 and AI and in total.

	Angle of pectoral muscle (in degrees)
	≤20°	>20°
	% (n=)	% (n=)
R1	35.3% (55)	64.7% (101)
R2	32.1% (50)	67.9% (106)
AI	41.7% (65)	58.3% (91)
	Agreement	Disagreement	Kappa
R1 and R2	78.8% (123)	21.2% (33)	0.53
R1 and AI	64.1% (100)	35.9% (56)	0.25
R2 and AI	67.3% (105)	32.7% (51)	0.31
R1, R2 and AI	55.1% (86)	44.9% (70)	0.35

	Fold in pectoral muscle
	No	Yes
	% (n=)	% (n=)

R1	66.7% (104)	33.3% (52)
R2	63.5% (99)	36.5% (57)
AI	87.2% (136)	12.8% (20)
	Agreement	Disagreement	Kappa
R1 and R2	86.5% (135)	13.5% (21)	0.70
R1 and AI	73.1% (114)	26.9% (42)	0.28
R2 and AI	67.3% (105)	32.7% (51)	0.18
R1, R2 and AI	63.5% (99)	36.5% (57)	0.39

	Pectoral muscle to posterior nipple line
	Sufficient	Insufficient
	% (n=)	% (n=)

R1	75.6% (118)	24.4% (38)
R2	70.5% (110)	29.5% (46)
AI	62.2% (97)	37.8% (59)
	Agreement	Disagreement	Kappa
R1 and R2	92.3% (144)	7.7% (12)	0.81
R1 and AI	69.9% (109)	30.1% (47)	0.31
R2 and AI	67.3% (105)	32.7% (51)	0.27
R1, R2 and AI	64.7% (101)	35.3% (55)	0.45

AI: artificial intelligence.

The percentage of images with a “straight pectoral muscle” varied from 71.8% (112/156) to 75.0% (117/156), while concave pectoral muscle varied from 16.0% (25/156) to 19.2% (30/156), and convex from 5.8% (9/156) to 12.2% (19/156) (Table 5). Slight agreement was observed between R1 and AI (κ = 0.06), and between R2 and AI (κ = 0.20). The percentage of images with an “inframammary fold that was open and free of skin folds” varied between 38.5% (60/156) and 53.8% (84/156) for R1, R2 and AI, while open with skin fold varied between 32.1% (50/156) and 41.0% (64/156), and inframammary fold not included in the image between 9.0% (14/156) and 20.5% (32/156) (Table 5). Fair agreement was observed between R1 and AI (κ = 0.34), and between R2 and AI (κ = 0.39).

Table 5.

Percentage (%) and proportion (n = out of 156 examinations) of shape of pectoral muscle and inframammary fold visibility for MLO view, by R1, R2 and AI, and the agreement, disagreement and Cohen’s Kappa value between each R1, R2 and AI and in total.

	Shape of pectoral muscle
	Straight	Concave	Convex
	% (n=)	% (n=)	% (n=)
R1	71.8% (112)	17.3% (27)	10.9% (17)
R2	71.8% (112)	16.0% (25)	12.2% (19)
AI	75.0% (117)	19.2% (30)	5.8% (9)
	Agreement	Disagreement	Kappa
R1 and R2	82.1% (128)	17.9% (28)	0.60
R1 and AI	60.3% (94)	39.7% (62)	0.06
R2 and AI	66.0% (103)	34.0% (53)	0.20
R1, R2 and AI	54.5% (85)	45.5% (71)	0.29
	Inframammary fold visibility
	Open and free of skin folds	Open with skin fold	Not included
	% (n=)	% (n=)	% (n=)
R1	38.5% (60)	41.0% (64)	20.5% (32)
R2	46.8% (73)	32.1% (50)	19.2% (30)
AI	53.8% (84)	37.2% (58)	9.0% (14)
	Agreement	Disagreement	Kappa
R1 and R2	78.2% (122)	21.8% (34)	0.66
R1 and AI	59.0% (92)	41.0% (64)	0.34
R2 and AI	62.8% (98)	37.2% (58)	0.39
R1, R2 and AI	50.6% (79)	49.4% (77)	0.46

AI: artificial intelligence.

Discussion

This study aimed to determine the agreement between an AI system and radiographers in the assessment of breast positioning criteria. We observed a substantial to almost perfect agreement between AI and the radiographers for the nipple in profile criterion and an excellent correlation for pectoral nipple line. However, there was only a slight to moderate agreement for the other criteria, and generally a higher agreement between the two pairs of radiographers than between the radiographers and AI.

The reasons for the higher agreement and correlation for the criteria nipple in profile and pectoral nipple line, compared to the other criteria, are not obvious to the authors. Perhaps these criteria are simply easier to agree upon based on the appearance in the image – however, agreement between the pairs of radiographers was as high or even higher for other criteria. AI relies on premises set by humans to provide reference standards for the criteria.¹⁸ In this case, premises were set by other radiographers than those participating in this study. It is possible that those who provided reference standards for this AI interpreted those two criteria in the same way as the radiographers in our study did. The lower agreement observed between the radiographers and AI on the other positioning criteria might reflect differences in the interpretation or definition of the criteria or the assessment methods. For instance, some criteria, like whether or not a skin fold is present, might be interpreted differently based on whether the fold is transparent and whether it covers any anatomy of interest. Furthermore, the assessment method will obviously impact on the agreement; for instance, how to assess rotation of the breast, pectoral nipple line, and criteria related to appearance of pectoral muscle.

The agreement between the two pairs of radiographers was generally higher than the agreement between R1 and AI, R2 and AI, and combined. A multi-centre study by Sharma et al.²² evaluated agreement between one radiologist and nine radiographers in assessing positioning errors on 672 rejected DM examinations. The assessors were provided with similar criteria to those included in the present study. They reported slight to moderate agreement between the assessors (κ = 0.09–0.49), and thus the observed agreement between the radiographers in our study was higher than in their study.²² The high agreement observed in our study could be due to the chosen method of using a plenary assessment of ten images before starting the assessment for the study and two radiographers discussing the criteria. However, we would argue that our results are promising when it comes to having a common understanding of breast positioning criteria and assessment methods. Our results support using training of radiographers to achieve a more uniform assessment, which could further lead to optimal mammographic positioning.²³

AI has great potential in mammographic screening by reducing the subjectivity in the image quality and breast positioning assessment among radiographers. An objective and reproducible assessment at the screening examination might reduce retakes due to preferences or opinions among the radiographers, and reduce recalls due to inadequate breast positioning. AI could also reduce the time radiographers spend assessing images, free up resources, and make the screening programs more efficient and cost-effective. However, if the radiographers do not understand, agree with or trust the performance of AI systems, these potential advantages could be lost. This highlights the need for a common understanding and definition of the criteria and assessment methods, in addition to transparency around how the AI systems work. In the future, AI may also have a significant impact on the radiologist’s screen reading.²⁴ If AI becomes the sole reader of a proportion of the mammograms in a screening program, it is particularly important that there is common understanding of adequate image quality and breast positioning to ensure reproducible imaging over time. Today, there is limited information available about how different AI systems work. Transparency seems to be crucial in the process of successfully adapting AI systems into clinical practice.

Several publications have emphasized the need for a reliable, valid and objective system of image quality assessment in mammography.^11,13 The current systems for image quality assessment were developed in a different era, without the same technical and digital possibilities. Today, image quality can be assessed directly after every exposure, large and advanced detectors with better dynamic range are available, and there are many post-processing options. Thus, the relevance of the image criteria might have changed since the original criteria were identified. Furthermore, studies have raised questions regarding the achievability and relevance of some of the criteria used.^2,12,25 The most recent attempts to improve systems for image quality assessment have, as far as we are aware, not reduced the subjectivity or lack of evidence.^5,26 This highlights the importance of standardized and evidence-based criteria in systems for assessment of breast positioning in order to achieve uniform assessment and optimal image quality in mammography.

We assume that the criteria selected for this study impacted the results. Our study included fewer criteria than used in clinical practice in Norway⁹ and in other studies.²⁶ For instance, we included three criteria for CC view; however, there is limited evidence and consistency about the choice of CC criteria.²⁷ Our study solely assessed breast positioning criteria, but other criteria related to blur, exposure and post-processing adequacy are also of importance for image quality. Furthermore, we only assessed selected positioning criteria and no overall image quality or technical errors. However, the choice of criteria was limited by available output from the AI system evaluated. Also, we excluded images of women with breast implants, which might represent a limitation of the study.

Conclusion

AI has great potential in image quality and breast positioning assessment in mammographic screening by reducing subjectivity. However, we observed varying agreement between the radiographers and AI for several breast positioning criteria, and there was higher agreement between the radiographers. Standardized and evidence-based definitions and assessment methods are needed to reach optimal image quality in mammography.

Supplemental Material

sj-pdf-1-msc-10.1177_0969141321998718 - Supplemental material for Assessment of breast positioning criteria in mammographic screening: Agreement between artificial intelligence software and radiographers

Supplemental material, sj-pdf-1-msc-10.1177_0969141321998718 for Assessment of breast positioning criteria in mammographic screening: Agreement between artificial intelligence software and radiographers by Gunvor G Waade, Anders Skyrud Danielsen, Åsne S Holen, Marthe Larsen, Berit Hanestad, Nina-Merete Hopland, Vanya Kalcheva and Solveig Hofvind in Journal of Medical Screening

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Anders Skyrud Danielsen

Solveig Hofvind

Supplemental material

Supplemental material for this article is available online.

References

Bassett

Farria

Bansal

, et al. Reasons for failure of a mammography unit at clinical image review in the American College of Radiology Mammography Accreditation Program. Radiology 2000; 215: 698–702.

Taplin

Rutter

Finder

, et al. Screening mammography: clinical image quality and the risk of interval breast cancer. AJR Am J Roentgenol 2002; 178: 797–803.

Landsveld-Verhoeven

The right focus; manual on mammography positioning technique. Nijmegen: LRCB, 2015.

Mercer

Hill

Kelly

, et al. Practical mammography in digital mammography: a holistic approach (Hogg P, Kelly J, Mercer C, eds.), 2015, pp. 175–188. DOI: 10.1007/978-3-319-04831-4_21.

NHS Breast Screening Programme. Guidance for breast screening mammographers. 3rd ed. UK: Public Health England, 2017, Updated 2020, www.gov.uk/government/publications/breast-screening-quality-assurance-for-mammography-and-radiography/guidance-for-breast-screening-mammographers (accessed 18 February 2021).

EUREF European Guidelines. European guidelines for quality assurance in breast cancer screening and diagnosis. 4th ed. 2006. Luxembourg: Office for Official Publications of the European Communities. https://www.euref.org/european-guidelines/4th-edition

Breast Imaging Clinical Education Review Committee, BreastScreen NSW. Research proposal: evaluation of mammogram quality. A comparison of 2 image classification systems. BreastScreen NSW, Sydney: State Co-ordination Unit, 2001.

The National Health Service Breast Screening Programme. Quality Assurance Guidelines for Radiographers. NHSBSP Publication no. 30. Sheffield: NHS Cancer Screening Programmes, 1994.

Vee

Gullien

Handberg

, et al. Chapter 5: Directions for radiographers in the quality assurance manual of the Norwegian Breast Cancer Screening Program (NBCSP). Oslo: The Cancer Registry of Norway, Institute of Population-Based Cancer Research, 2011, www.kreftregisteret.no/globalassets/publikasjoner-og-rapporter/mammografiprogrammet/kval-man-radiograf_v1.0_innholdsfortegnelse.pdf

10.

Hofvind

Vee

Sørum

, et al. Quality assurance of mammograms in the Norwegian breast cancer screening program. Eur J Radiogr 2009; 1: 22–29.

11.

Moreira

Svoboda

Poulos

, et al. Comparison of the validity and reliability of two image classification systems for the assessment of mammogram quality. J Med Screen 2005; 12: 38–42.

12.

Spuur

Webb

Poulos

, et al. Mammography image quality and evidence based practice: analysis of the demonstration of the inframammary angle in the digital setting. Eur J Radiol 2018; 100: 76–84.

13.

Boyce

Gullien

Parashar

, et al. Comparing the use and interpretation of PGMI scoring to assess the technical quality of screening mammograms in the UK and Norway. Radiography 2015; 21: 342–347.

14.

Bülow

Meetz

Kutra

, et al. Automatic assessment of the quality of patient positioning in mammography. SPIE Med Imag 2013; 8670: 867024. DOI: 10.1117/12.2007980.

15.

Moran

Conci

Rêgo

, et al. Techniques for automated analysis of mammography positioning failures. ECR 2018; 2018: 1089. DOI: 10.1594/ecr2018/C-1089. https://epos.myesr.org/poster/esr/ecr2018/C-1089

16.

Wang

Ross

Khan

, et al. A validation study of automated mammographic breast positioning metrics. ECR 2016 2016; 0854. DOI: 10.1594/ecr2016/C-0854. https://epos.myesr.org/poster/esr/ecr2016/C-0854

17.

Johnston

Hill

Wang

, et al. Determination of adequate breast tissue visualization using an automated posterior nipple line measure. ECR 2017; 2017: 1029. https://epos.myesr.org/poster/esr/ecr2017/C-1029

18.

Whelehan

Clinical image quality in mammography. RAD Mag 2016; 493: 16–17.

19.

Cancer in Norway 2016. Special Issue: The Norwegian Breast Cancer Screening Program, 1996-2016: celebrating 20 years of organised mammographic screening (Hofvind S, ed.). Oslo: Cancer Registry of Norway, 2017, www.kreftregisteret.no/globalassets/cancer-in-norway/2016/mammo_cin2016_special_issue_web.pdf

20.

Koo

MY.

A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.

21.

Landis

Koch

GG.

The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.

22.

Sharma

Schofield

Fletcher

, et al. Mammography positioning errors: a multi-centre study. ECR 2019.

23.

Pal

Ikeda

Jesinger

, et al. Improving performance of mammographic breast positioning in an academic radiology practice. AJR Am J Roentgenol 2018; 210: 807–815.

24.

Sechopoulos

Teuwen

Mann

Artificial intelligence for breast cancer detection in mammography and digital breast tomosynthesis: state of the art. Semin Cancer Biol 2020; in press. DOI: 10.1016/j.semcancer.2020.06.002.

25.

Guertin

Theberge

Dufresne

, et al. Clinical image quality in daily practice of breast cancer mammography screening. Can Assoc Radiol J 2014; 65: 199–206.

26.

Taylor

Parashar

Bouverat

, et al. Mammographic image quality in relation to positioning of the breast: a multicentre international evaluation of the assessment systems currently used, to provide an evidence base for establishing a standardised method of assessment. Radiography (Lond) 2017; 23: 343–349.

27.

Sweeney

Lewis

Hogg

, et al. A review of mammographic positioning image quality criteria for the craniocaudal projection. Br J Radiol 2018; 91: 20170611.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.32 MB