Deep learning for differentiating benign from malignant tumors on breast-specific gamma image

Abstract

BACKGROUND:

Breast diseases are a significant health threat for women. With the fast-growing BSGI data, it is becoming increasingly critical for physicians to accurately diagnose benign as well as malignant breast tumors.

OBJECTIVE:

The purpose of this study is to diagnose benign and malignant breast tumors utilizing the deep learning model, with the input of breast-specific gamma imaging (BSGI).

METHODS:

A benchmark dataset including 144 patients with benign tumors and 87 patients with malignant tumors was collected and divided into a training dataset and a test dataset according to the ratio of 8:2. The convolutional neural network ResNet18 was employed to develop a new deep learning model. The model proposed was compared with neural network and autoencoder models. Accuracy, specificity, sensitivity and ROC were used to evaluate the performance of different models.

RESULTS:

The accuracy, specificity and sensitivity of the model proposed are 99.1%, 98.8% and 99.3% respectively, which achieves the best performance among all methods. Additionally, the Grad-CAM method is used to analyze the interpretability of the diagnostic results based on the deep learning model.

CONCLUSION:

This study demonstrates that the proposed deep learning method could help physicians diagnose benign and malignant breast tumors quickly as well as reliably.

1. Introduction

Breast diseases are a significant health threat for women. According to data from the World Health Organization, more than two million women worldwide suffer from breast cancers each year. Approximately more than 600,000 women die of breast cancers every year, which account for 15% of female cancer deaths [1]. According to the 2010 China breast disease survey, breast cancers have become the most common cancer threat for women living in cities, whose incidence rate has an increasing trend year by year. Clinical practice shows that early screening and diagnoses are significantly important to breast diseases.

Current breast cancer screening methods include mammogram (MMG), breast ultrasound (BUS) and MRI. Although through these examinations, early detection rates can be improved and breast cancer-related mortality can be reduced, these methods still have limitations [1]. The accuracy of MMG is affected by the density of breast tissues. The higher the density of the breast tissues is, the lower the diagnostic precision will be. The BUS image resolution is very low, making it difficult to find tiny sick areas. During a breast MRI examination, there will be varying degrees of background enhancement in the normal breast tissues around tumors, which lowers the diagnosis accuracy.

Breast-specific gamma imaging (BSGI) is a nuclear medical device in which a gamma camera is used for the functional molecular imaging of breasts. It is used to detect breast cancers that are difficult to diagnose via MMG. The metabolic function of tumors changes earlier than the anatomical changes [2, 3, 4, 5]. 99mTc-MIBI radiopharmaceutical is used in BSGI to specifically combine with intracellular mitochondria, which can provide a diagnostic basis by identifying the differences in mitochondrial metabolic activity to distinguish malignant tumors. At the same time, unlike MMG, BSGI is not affected by breast tissue density, nor is it affected by implants, distorted tissue morphology or scars caused by previous operations. Previous studies have shown that BSGI has a higher sensitivity and specificity to breast cancers than mammography, ultrasound and MRI [6, 7, 8]. With the continuous improvement of its technical level, BSGI detection has been gradually developed.

Recently, with the vigorous development of artificial intelligence and computer computing power, deep learning technology has also achieved remarkable success. Among them, convolutional neural network (CNN) models have been successfully and widely applied, studies on whose application in the medical imaging field are also fruitful. Encouraging achievements have been achieved in molecular imaging in the field of nuclear medicine [2, 9, 10, 11]. Unlike traditional neural network methods, through convolutional neural networks, features can be automatically extracted through convolution operations to accomplish end-to-end learning tasks [9, 10, 11]. Currently, there have been reports to which researchers adopted convolutional neural network models to assist the diagnosis via mammary gland mammography and MRI images [1, 3]. To our knowledge, there is no research in which deep learning is applied to diagnosis BSGI images. The innovation of this paper is that, for the first time, a deep learning model based on convolutional neural networks is developed to differentiate benign and malignant breast tumors on BSGI images. The performance of the method proposed is compared with that of other machine learning methods.

2. Materials and method

2.1 Image data

A total of 231 patients with breast tumors were enrolled at Weihai Maternal and Children Health Hospital and Weihai Municipal Hospital from June 2020 to July 2021. All patients were female with an average age of 53.2, ranging from 28 to 73. There were 144 patients with benign tumors and 87 with malignant tumors. The benign or malignant diagnoses came from pathological results. The study was approved by the Weihai Maternal and Children Health Hospital Ethics Committees (2019014) and Weihai Municipal Hospital Ethics Committees (2020026), which was in accordance with the principles of the Declaration of Helsinki. All patients had signed an informed consent.

BSGI images of bilateral breasts were collected from all patients. After 740MBq Technetium-99m methoxyisobutylisonitrile was injected into the vein of the upper limb, the patients waited for 10 minutes and BSGI images were collected. BSGI images collected from each breast contained two images, namely a craniocaudal-position (CC) and a mediolateral-oblique-position (MLO) image. The images of the left breast were recorded as LCC and LMLO, and similarly, RCC and RMLO referred to those from the right.

2.2 Deep learning method

The four images LCC, RCC, LMLO and RMLO were unified to the same size of 256 $\times$ 256, which were combined into a four-channel image, as is illustrated in Fig. 1.

Figure 1.

Four images of BSGI are combined into an image with four channels.

In this dataset, 80% of the data is selected as the training dataset and 20% as the test dataset. That is, BSGI images of 116 benign tumors and 70 malignant tumors are used for deep learning model training. After the training process, BSGI images of 28 benign tumors and 17 malignant ones are used to evaluate model performance.

Figure 2.

Network architecture to differentiate benign tumors from malignant ones.

A deep learning model is developed to differentiate benign and malignant tumors on BSGI images. ResNet18 is used as the core module of the developed model, as is shown in Fig. 2. Three ResNet18 (ResBlock) and two fully-connected (FC) layers are used. The input of the model proposed is a four-channel BSGI image, and its output is the diagnosis of benign or malignant tumors.

The migration learning strategy is adopted to the model proposed. The network weights of ResNet18 are pre-trained through the ImageNet dataset, which are used as the initial model weights. The network is implemented with PyTorch using Python 3.7. A SGD optimizer is used, whose weight attenuation parameter is 5 $\times$ 10 ${}^{-4}$ and the momentum parameter is 0.9. The classification cross-entropy loss is used to train the network model, with the batch size set as 8 and the learning rate set as 0.001.

In order to evaluate the performance of different machine learning methods, accuracy, specificity and sensitivity are used as metrics, meanwhile the performance of the deep learning method proposed is compared with that of the autoencoder method [12] and the neural network method [13].

3. Results and discussion

Table 1

Comparison of the performance of different methods to differentiate benign tumors from malignant ones on BSGI images

Method	Performance
Neural network	Accuracy	0.802
	Specificity	0.747
	Sensitivity	0.836
Autoencoder	Accuracy	0.889
	Specificity	0.859
	Sensitivity	0.907
Proposed	Accuracy	0.991
	Specificity	0.988
	Sensitivity	0.993

Figure 3.

Performance comparison graph.

The same training dataset and test dataset are used to evaluate three different machine learning methods. The experiment on each method is carried out 5 times, and the average indicators are taken as the results, which are shown in Table 1, and the graphs are shown in Fig. 3. The accuracy, specificity and sensitivity of the method proposed increase by 0.189, 0.141 and 0.157 respectively compared with the neural network method and 0.102, 0.129 as well as 0.086 respectively compared with the autoencoder method. The method proposed has the best performance in distinguishing benign tumors from malignant ones. The results of ${\chi }^2$ tests of paired comparison show that the method proposed is significantly different from the other two methods in sensitivity, specificity and accuracy ( $p<$ 0.05).

Figure 4.

ROC and AUC of different methods.

The ROC curve is also used for performance comparison, as is illustrated in Fig. 4. The results are consistent with the performance indicators mentioned above. The AUC is also calculated and shown in the figure. The AUC of the method proposed is 0.98, which is 0.09 and 0.20 higher than that of the neural network method and the autoencoder method respectively.

The convolutional neural network model proposed is a black-box model in which the processing and diagnosis process cannot be inferred. To figure out how the model assists in the diagnosis, the Grad-CAM method [14, 15, 16, 17] is used in this study to perform a visual analysis on the classification of model diagnoses. Grad-CAM has been successfully used to provide the interpretability of the model for medical images, such as MRI and endoscopic images [16, 17]. By providing a heat map of the focus area of the final model classifier, it helps to understand the discrimination process of the model. The visualization results of benign and malignant tumors are shown in Fig. 5, in which the tumor positions are marked. The focus of the classification model on malignant lesions is mainly the tumors and their vicinities, whereas the focus of benign lesions is mainly on the blank background areas of the images. This result enhances the model interpretability and improves the trust of physicians in the discriminating ability of the model.

Figure 5.

Visual result of deep-learning-based tumor classification of BSGI.

Currently, deep learning methods have been applied in many studies to the diagnosis of breast diseases, which have achieved good results [1, 2]. Using a convolutional neural network to quickly diagnose breast diseases via mammary mammography images, an accuracy of 97.13% can be achieved. Using a convolutional neural network to identify the nature of tumors through breast ultrasound images, a result with a sensitivity of 90.3% and a specificity of 79.4% is yielded. At present, the use of deep learning in the diagnosis of breast diseases is mainly focused on mammography images, breast MRI images and breast ultrasound images. There is no report on the application of deep learning methods to BSGI images. In this research, the application of deep learning methods based on convolutional neural networks to BSGI images is initially explored, and the results show that this method is feasible and effective. Deep learning has great diagnostic capabilities, and it is possible to help professional physicians perform an auxiliary diagnosis of BSGI using deep learning.

BSGI images contain enough information to distinguish malignant tumors clinically. Although the CNN network used in this paper is simple enough, a high accuracy, specificity and sensitivity are obtained, which is consistent with the effect of BSGI in clinical practice. On the other hand, the method of transfer learning is adopted in this paper. Using the large-scale image database ImageNet, the network module is trained to obtain the weights, which contain rich image features, so through fine-tuning training in BSGI images, the desired network training effect can be obtained. With the help of these strategies, CNNs have been widely used in small datasets of medical images, and the identification of benign and malignant tissues is also no exception. For example, samples containing 158 benignant tumors and 32 malignant ones were used as reference [18], 100 regions of 35 patients were classified in reference [19], and a total of 26 benignant masses as well as 30 malignant ones were distinguished in reference [20].

There are limitations in this study. The model used in this study has good diagnostic performance for benign and malignant breast tumors. However, the total sample size of this study is relatively small, and more samples are required to verify the effectiveness of the model. Furthermore, this study is limited to cases in Weihai City, China, and cases in more regions need to be carried out in the future.

4. Conclusions

Among all the machine learning models compared, the deep learning model demonstrated the best performance in the diagnosis of benign and malignant breast tumors. With the fast-growing BSGI data, it is becoming increasingly critical for physicians to accurately and efficiently diagnose benign and malignant breast tumors. With the progress of deep learning technology in the medical field, it would be a natural choice for physicians to apply deep-learning-based methods to assist themselves in BSGI images. The study demonstrates that the use of the deep learning method can possibly help physicians diagnose benign and malignant breast tumors quickly as well as reliably.