Pulmonary micronodules account for 80% of all lung nodules. Generally, pulmonary micronodules in the early stages can be detected on thoracic computed tomography (CT) scans. Early diagnosis is crucial for improving the patient’s survival rate.
OBJECTIVE:
This paper aims to estimate the malignancy risk of pulmonary micronodules and potentially improve the survival rate.
METHODS:
We extract 3D features of the CT images to obtain richer characteristics. Because superior performance can be achieved by having deep layers, we apply a 3D residual network (3D-ResNet) to classify the pulmonary micronodule. We construct a framework by using three parallel ResNets whose inputs are CT images in different regions of interest, i.e., the multiview of the image. To further evaluate the applicability of the framework, we make a five-category classification and achieve good performance.
RESULTS:
By fusing different characteristics from three views, we achieve the area under the receiver operating characteristic curve (AUC) of 0.9681. Based on the results of the experiments, our 3D-ResNet has a better performance than 3D-VGG and 3D-Inception in terms of precision (the increase rates are 13.7% and 7.4%), AUC (the increase rates are 15.8% and 5.3%), and accuracy (the increase rates are 14.3% and 4.5%). Meanwhile, the recall performance is close to that of the 3D-Inception network.
CONCLUSION:
Overall, the framework we propose has applicability and feasibility in pulmonary micronodule classification.
BaiC.ChoiC.ChuC.M., Evaluation of pulmonary nodules: Clinical practice consensus guide-lines for Asia, Chest150(4) (2016), 877–893. doi:10.1016/j.chest.2016.02.650.
2.
MsaouelP.PissimissisN.HalapasA., Mechanisms of bone metastasis in prostate cancer: Clinical implications, Best Pract. Res. Clin. Endoc. Metab.22(2) (2008), 341–355. doi:10.1016/j.beem.2008.01.011.
3.
National Lung Screening Trial Research Team. The national lung screening trial: Overview and study design, Radiology258(1), 2011 243–253. doi:10.1148/radiol.10091808.
4.
McWilliamsA.TammemagiM.C.MayoJ.R., Probability of cancer in pulmonary nodules detected on first screening CT, N. Engl. J. Med.369(10) (2013), 910–919. doi:10.1056/NEJMoa1214726.
5.
SwensenS.J.SilversteinM.D.IlstrupD.M., The probability of malignancy in solitary pulmonary nodules: Application to small radiologically indeterminate nodules, Archives of Internal Medicine157(8) (1997), 849–855. doi:10.1001/archinte.157.8.849.
6.
MacMahonH.AustinJ.H.M.GamsuG., Guidelines for management of small pulmonary nodules detected on CT scans: A statement from the Fleischner Society, Radiology237(2) (2005), 395–400. doi:10.1148/radiol.2372041887.
7.
SpitzM.R.HongW.K.AmosC.I., A risk model for prediction of lung cancer, JNCI-J. Natl. Cancer Inst.99(9) (2007), 715–726. doi:10.1093/jnci/djk153.
8.
ToewsM. and ArbelT., A statistical parts-based model of anatomical variability, IEEE Trans. Med. Imaging26(4) (2007), 497–508. doi:10.1109/TMI.2006.895907.
9.
SeffA.LuL.CherryK.M.2D view aggregation for lymph node detection using a shallow hierarchy of linear classifiers, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer International Publishing, 2014, pp. 544–552. doi:10.1007/978-3-319-10404-1_68.
10.
WuD.LuL. and BiJ., Stratified learning of local anatomical context for lung nodules in CT images, in: Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, IEEE, 2010, pp. 2791–2798. doi:10.1109/CVPR.2010.5540008.
11.
ChangC.C. and LinC.J., LIBSVM: A library for support vector machines, ACM Trans. Intell. Syst. Technol.2(3) (2011), 27. doi:10.1145/1961189.1961199.
12.
FriedmanJ.H., Greedy function approximation: A gradient boosting machine, Ann. Stat.29 (2001), 1189–1232. doi:10.1214/aos/1013203451.
13.
BreimanL., Random forests, Mach. Learn.45(1) (2001), 5–32. doi:10.1023/A:1010933404324.
14.
XieY.ZhangJ.XiaY.FulhamM. and ZhangY., Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT, Inf. Fusion42 (2018), 102–110. doi:10.1016/j.inffus.2017.10.005.
15.
ShenW.ZhouM.YangF.YangC. and TianJ., Multi-scale convolutional neural networks for lung nodule classification, in: International Conference on Information Processing in Medical Imaging, Vol. 24, , Springer, 2015, pp. 588–599. doi:10.1007/978-3-319-19992-4_46.
16.
HanF.F.WangH.F.ZhangG.P., Texture feature analysis for computer-aided diagnosis on pulmonary nodules, J. Digit. Imaging28(1) (2015), 99–115. doi:10.1007/s10278-014-9718-8.
17.
DharaA.K.MukhopadhyayS.DuttaA.GargM. and KhandelwalN., A combination of shape and texture features for classification of pulmonary nodules in lung CT images, J. Digit. Imaging29 (2016), 466–475. doi:10.1007/s10278-015-9857-6.
18.
KrizhevskyA.SutskeverI. and HintonG.E., Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, Curran Associates, Inc., 2012, pp. 1097–1105.
19.
ArevaloJGonzálezF.A.Ramos-PollánR., Representation learning for mammography mass lesion classification with convolutional neural networks, Comput. Meth. Programs Biomed.127 (2016), 248–257. doi:10.1016/j.cmpb.2015.12.014.
20.
FaustO.AcharyaR.NgE.Y.K., Algorithms for the automated detection of diabetic retinopathy using digital fundus images: A review, J. Med. Syst.36(1) (2012), 145–157. doi:10.1007/s10916-010-9454-7.
21.
HaloiM., Improved micro aneurysm detection using deep neural networks, arXiv preprint arXiv:1505.04424, 2015.
22.
KamnitsasK.LedigC.NewcombeV.F.J., Efficient multiscale 3D CNN with fully connected CRF for accurate brain lesion segmentation, Med. Image Anal.36 (2017), 61–78. doi:10.1016/j.media.2016.10.004.
23.
HavaeiM.DavyA.Warde-FarleyD., Brain tumor segmentation with deep neural networks, Med. Image Anal.35 (2017), 18–31. doi:10.1016/j.media.2016.05.004.
24.
Hosseini AslE.GimelfarbG. and El-BazA., Alzheimer’s disease diagnostics by a deeply supervised adaptable 3D convolutional network, Front. Biosci.23 (2016), 584–596. doi:10.2741/4606.
25.
HuC.JuR.ShenY.Clinical decision support for Alzheimer’s disease based on deep learning and brain network, in: Communications (ICC), 2016 IEEE International Conference on, IEEE, 2016, pp. 1–6. doi:10.1109/ICC.2016.7510831.
26.
CiompiF.de HoopB.van RielS.J., Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the box, Med. Image Anal.26(1) (2015), 195–202. doi:10.1016/j.media.2015.08.001.
SzegedyC.LiuW.JiaY.Going deeper with convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2015, pp. 1–9. doi:10.1109/CVPR.2015.7298594.
30.
SimonyanK. and ZissermanA., Very deep convolutional networks for large-scale image recognition, in: International Conference on Learning Representations, Ithaca, NY: arXiv.org, 2015.
31.
IoffeS. and SzegedyC., Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: International Conference on Machine Learning, JMLR.org, 2015, pp. 448–456.
32.
GlorotX. and BengioY., Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2010, pp. 249–256.
33.
VeitA.WilberM. and BelongieS., Residual networks are exponential ensembles of relatively shallow networks, arXiv preprint arXiv:1605.06431, 2016.
34.
SrivastavaR.K.GreffK. and SchmidhuberJ., Training very deep networks, in: Advances in Neural Information Processing Systems, Curran Associates, Inc., 2015, pp. 2377–2385.
35.
HeK.ZhangX.RenS. and SunJ., Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. doi:10.1109/CVPR.2016.90.
36.
RuczinskiI.KooperbergC. and LeBlancM., Logic regression, J. Comput. Graph. Stat.12(3) (2003), 475–511.
37.
AhaD.W., Special Al review issue on lazy learning, Artif. Intell. Rev.11 (1997).
38.
YeH.LuoF. and XuY., Application of RBF network based on immune algorithm to predicting of wastewater treatment, in: International Symposium on Neural Networks, Springer, 2009, pp. 1197–1202. doi:10.1007/978-3-642-01513-7_133.
39.
HintonG.E.OsinderoS. and TehY.W., A fast learning algorithm for deep belief nets, Neural Comput.18(7) (2006), 1527–1554. doi:10.1162/neco.2006.18.7.1527.
40.
SzegedyC.IoffeS.VanhouckeV. and AlemiA.A., Inception-v4, inception-resnet and the impact of residual connections on learning, in: National Conference on Artificial Intelligence, vol. 4, 2016, pp. 4278–4284.
41.
JarrettK.KavukcuogluK. and LeCunY., What is the best multi-stage architecture for object recognition? in: 2009 IEEE 12th International Conference on Computer Vision, IEEE, 2009, pp. 2146–2153. doi:10.1109/ICCV.2009.5459469.
42.
GlorotX.BordesA. and BengioY., Deep sparse rectifier neural networks, in: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2011, pp. 315–323.
43.
KingmaD.P. and BaJ., Adam: A method for stochastic optimization, in: International Conference on Learning Representations, Ithaca, NY: arXiv.org, 2015.
44.
FawcettT., An introduction to ROC analysis, Pattern Recognit. Lett.27(8) (2006), 861–874. doi:10.1016/j.patrec.2005.10.010.
45.
ArmatoS.G.McLennanG.BidautL., The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans, Med. Phys.38(2) (2011), 915–931.
46.
LampertT.A.StumpfA. and GancarskiP., An empirical study into annotator agreement, ground truth estimation, and algorithm evaluation, IEEE Trans. Image Process.25(6) (2016), 2557–2572. doi:10.1109/TIP.2016.2544703.
47.
SermanetP.EigenD.ZhangX., Overfeat: Integrated recognition, localization and detection using convolutional networks, arXiv preprint arXiv:1312.6229, 2013.
48.
SrivastavaN.HintonG.KrizhevskyA., Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res.15(1) (2014), 1929–1958.
49.
BergstraJ.BreuleuxO.BastienF.Theano: A CPU and GPU Math Compiler in Python, in: Proc. 9th Python in Science Conf, 2010, pp. 1–7. doi:10.25080/Majora-92bf1922-003.
50.
BastienF.LamblinP.PascanuR., Theano: New features and speed improvements, arXiv preprint arXiv:1211.5590, 2012.
51.
BergstraJ.BastienF.BreuleuxO.Theano: Deep learning on gpus with python, in: NIPS 2011, BigLearning Workshop, vol. 3, Microtome Publishing, 2011.
52.
OchsR.KimH.J.AngelE., Forming a reference standard from LIDC data: Impact of reader agreement on reported CAD performance, Proceedings of the SPIE, Medical Imaging 2007: Computer-Adied Diagnosis6514 (2007). doi:10.1117/12.707916.
53.
KayaA. and Can.A.B., A weighted rule-based method for predicting malignancy of pulmonary nodules by nodule characteristics, J. Biomed. Inform.56 (2015), 69–79. doi:10.1016/j.jbi.2015.05.011.
54.
HancockM.C. and MagnanJ.F., Lung nodule malignancy classification using only radiologist-quantified image features as inputs to statistical learning algorithms: Probing the lung image database consortium dataset with two statistical learning methods, J. Med. Imaging3(4) (2016). doi:10.1117/1.JMI.3.4.044504.
