Fully Automated Neural Network Framework for Pulmonary Nodules Detection and Segmentation

Abstract

Lung cancer is the leading cause of cancer death worldwide, and most patients are diagnosed with advanced stages for lack of symptoms in the early stages of the disease, leading to poor prognosis. It is thus of great importance to detect lung cancer in the early stages which can reduce mortality and improve patient survival significantly. Although there are many computer aided diagnosis (CAD) systems used for detecting pulmonary nodules, there are still few CAD systems for detection and segmentation, and their performance on small nodules is not ideal. Thus, in this paper, we propose a deep cascaded multitask framework called mobilenet split-attention Yolo unet, the mobilenet split-attention Yolo(Msa-yolo) greatly enhance the feature of small nodules and boost up their performance, the overall result shows that the mean accuracy precision (mAP) of our Msa-Yolo compared to Yolox has increased from 85.10% to 86.64% on LUNA16 dataset, and from 90.13% to 94.15% on LCS dataset compared to YoloX. Besides, we get only 8.35 average number of candidates per scan with 96.32% sensitivity on LUNA16 dataset, which greatly outperforms other existing systems. At the segmentation stage, the mean intersection over union (mIOU) of our CAD system has increased from 71.66% to 76.84% on LCS dataset comparing to baseline. Conclusion: A fast, accurate and robust CAD system for nodule detection, segmentation and classification is proposed in this paper. And it is confirmed by the experimental results that the proposed system possesses the ability to detect and segment small nodules.

Keywords

Pulmonary nodule deep learning medical segmentation

Get full access to this article

View all access options for this article.

References

S.G.

Armato ,

McLennan ,

L.M.

Bidaut ,

M.F.

McNitt-Gray ,

C.R.

Meyer ,

A.P.

Reeves ,

Zhao ,

D.R.

Aberle ,

C.I.

Henschke ,

E.A.

Hoffman ,

E.A.

Kazerooni ,

MacMahon ,

E.J.R.

Van Beeke ,

D.F.

Yankelevitz ,

A.M.

Biancardi ,

P.H.

Bland ,

M.S.

Brown ,

R.M.

Engelmann ,

G.E.

Laderach ,

Max ,

R.C.

Pais ,

D.P.

Qing ,

R.Y.

Roberts ,

A.R.

Smith ,

Starkey ,

Batrah ,

Caligiuri ,

A.O.

Farooqi ,

G.W.

Gladish ,

C.M.

Jude ,

Munden ,

Petkovska ,

L.E.

Quint ,

L.H.

Schwartz ,

Sundaram ,

L.E.

Dodd ,

Fenimore ,

Gur ,

N.A.

Petrick ,

J.B.

Freymann ,

J.S.

Kirby ,

Hughes ,

A.V.

Casteele ,

Gupte ,

Sallamm ,

Heath ,

Kuhn ,

Dharaiya ,

Burns ,

Fryd ,

Salganicoff ,

Anand ,

Shreter ,

Vastagh and

B.Y.

Croft , The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans, Medical physics 38(2) (2011), 915–931. doi:10.1118/1.3528204.

Blandin Knight ,

P.A.

Crosbie ,

Balata ,

Chudziak ,

Hussell and

Dive , Progress and prospects of early detection in lung cancer, Open Biol 7(9) (2017). doi:10.1098/rsob.170070.

Bochkovskiy ,

C.-Y.

Wang and

H.-Y.M.

Liao , YOLOv4: Optimal speed and accuracy of object detection, arXiv: Computer Vision and Pattern Recognition, 2020.

Cao ,

Wu ,

Cao and

He , A comprehensive review of computer-aided diagnosis of pulmonary nodules based on computed tomography scans, IEEE Access 8 (2020), 154007–154023. doi:10.1109/ACCESS.2020.3018666.

L.-C.

Chen ,

Zhu ,

Papandreou ,

Schroff and

Adam , Encoder-decoder with atrous separable convolution for semantic image segmentation, in: Proceedings of the European conference on computer vision (ECCV), pp. 801–818.

Dey ,

Lu and

Hong , Diagnostic classification of lung nodules using 3D neural networks, in: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), IEEE, pp. 774–778. ISBN 1538636360.

Ding ,

Li ,

Hu and

Wang , Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2017, Springer International Publishing, pp. 559–567. ISBN 978-3-319-66179-7.

Dou ,

Chen ,

Yu ,

Qin and

P.-A.

Heng , Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection, IEEE Transactions on Biomedical Engineering 64(7) (2016), 1558–1567. doi:10.1109/TBME.2016.2613502.

Duan ,

Bai ,

Xie ,

Qi ,

Huang and

Tian , Centernet: Keypoint triplets for object detection, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 6569–6578.

10.

Ge ,

Liu ,

Wang ,

Li and

Sun , YOLOX: Exceeding YOLO series in 2021, arXiv:2107.08430, 2021.

11.

Gu ,

Tian and

Qi , Pulmonary nodules detection based on deformable convolution, IEEE Access 8 (2020), 16302–16309. doi:10.1109/ACCESS.2020.2967238.

12.

Howard ,

Sandler ,

Chu ,

L.-C.

Chen ,

Tan ,

Wang ,

Zhu ,

Pang and

Vasudevan , Searching for mobilenetv3, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 1314–1324.

13.

Huang ,

Lin ,

Tong ,

Hu ,

Zhang ,

Iwamoto ,

Han ,

Y.-W.

Chen and

Wu , Unet 3+: A full-scale connected unet for medical image segmentation, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp. 1055–1059. ISBN 1509066314.

14.

D.E.

Midthun , Early diagnosis of lung cancer, F1000, Prime Reports 5 (2013), 12.

15.

Monkam ,

Qi ,

Ma ,

Gao ,

Yao and

Qian , Detection and classification of pulmonary nodules using convolutional neural networks: A survey, IEEE Access 7 (2019), 78075–78091. doi:10.1109/ACCESS.2019.2920980.

16.

F.R.

Pereira ,

J.M.C.

De Andrade ,

D.L.

Escuissato and

L.F.

De Oliveira , Classifier ensemble based on computed tomography attenuation patterns for computer-aided detection system, IEEE Access 9 (2021), 123134–123145. doi:10.1109/ACCESS.2021.3109860.

17.

Ren ,

He ,

R.B.

Girshick and

Sun , Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (2015), 1137–1149. doi:10.1109/TPAMI.2016.2577031.

18.

Ronneberger ,

Fischer and

Brox , U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer, pp. 234–241.

19.

A.A.A.

Setio ,

Jacobs ,

Gelderblom and

van Ginneken , Automatic detection of large pulmonary solid nodules in thoracic CT images, Medical physics 42(10) (2015), 5642–5653. doi:10.1118/1.4929562.

20.

Su ,

Li and

Chen , Lung nodule detection based on faster R-CNN framework, Computer Methods and Programs in Biomedicine 200 (2021), 105866. doi:10.1016/j.cmpb.2020.105866.

21.

Tang ,

Ma ,

Li ,

Bai and

Chen , Classification of benign and malignant pulmonary nodules based on the multiresolution 3D DPSECN model and semisupervised clustering, IEEE Access 9 (2021), 43397–43410. doi:10.1109/ACCESS.2021.3060178.

22.

Wang ,

Wen ,

Lu ,

Niu ,

Pan and

Qian , Pulmonary nodule detection in volumetric chest CT scans using CNNs-based nodule-size-adaptive detection and classification, IEEE access 7 (2019), 46033–46044. doi:10.1109/ACCESS.2019.2908195.

23.

Wang ,

Chakraborty and

Chakraborty , 3D multi-scale DenseNet for malignancy grade classification of pulmonary nodules, in: 2020 11th International Conference on Awareness Science and Technology (iCAST), 2020, pp. 1–4.

24.

Wang ,

Feng ,

Chen ,

Lu ,

Chen ,

Yu ,

D.Z.

Chen and

Wu , Nodule-plus R-CNN and deep self-paced active learning for 3D instance segmentation of pulmonary nodules, IEEE Access 7 (2019), 128796–128805. doi:10.1109/ACCESS.2019.2939850.

25.

Wu ,

Liang ,

Liu and

Wen , Optimized HRNet for image semantic segmentation, Expert Systems with Applications 174 (2021), 114532. doi:10.1016/j.eswa.2020.114532.

26.

Wu ,

Tan ,

Chen and

Zhao , Decision based on big data research for non-small cell lung cancer in medical artificial system in developing country, in: Computer Methods and Programs in Biomedicine: An International Journal Devoted to the Development, Implementation and Exchange of Computing Methodology and Software Systems in Biomedical Research and Medical Practice, 2018.

27.

Wu ,

Chen ,

Yuan ,

Liu ,

Wang ,

Li and

Y.R.

Fu , Rethinking classification and localization for object detection, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 10183–10192.

28.

Yan ,

Pang ,

Qi ,

Zhu ,

Bai ,

Geng ,

Liu ,

Terzopoulos and

Ding , Classification of lung nodule malignancy risk on computed tomography images using convolutional neural network: A comparison between 2d and 3d strategies, in: Asian Conference on Computer Vision, Springer, pp. 91–101.

29.

Yu ,

Gao ,

Wang ,

Yu ,

Shen and

Sang , Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation, International Journal of Computer Vision 129(11) (2021), 3051–3068. doi:10.1007/s11263-021-01515-2.

30.

Zhang ,

Wu ,

Zhang ,

Zhu ,

Lin ,

Zhang ,

Sun ,

He ,

Mueller and

Manmatha , Resnest: Split-attention networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2736–2746.

31.

Zhang ,

Xia ,

Zeng and

Zhang , NODULe: Combining constrained multi-scale LoG filters with densely dilated 3D deep convolutional neural network for pulmonary nodule detection, Neurocomputing 317 (2018), 159–167. doi:10.1016/j.neucom.2018.08.022.

32.

Zhao ,

Shi ,

Qi ,

Wang and

Jia , Pyramid scene parsing network, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2881–2890.

33.

Zhao ,

Yan ,

Wolf and

Zhan , A deep-learning based automatic pulmonary nodule detection system, in: Medical Imaging 2018: Computer-Aided Diagnosis, Vol. 10575, SPIE, pp. 713–719.

34.

Zheng ,

Guo ,

Cui ,

R.N.J.

Veldhuis ,

Oudkerk and

P.M.A.

van Ooijen , Automatic pulmonary nodule detection in CT scans using convolutional neural networks based on maximum intensity projection, IEEE Transactions on Medical Imaging 39 (2020), 797–805. doi:10.1109/TMI.2019.2935553.

35.

Zhou ,

Gou ,

Tan and

Wu , A cascaded multi-stage framework for automatic detection and segmentation of pulmonary nodules in developing countries, IEEE Journal of Biomedical and Health Informatics (2022).

36.

Zhu ,

Liu ,

Fan and

Xie , Deeplung: Deep 3d dual path nets for automated pulmonary nodule detection and classification, in: 2018 IEEE winter conference on applications of computer vision (WACV), IEEE, pp. 673–681. ISBN 1538648865.