Traditional clinical diagnostic aid systems for medical images are facing challenges of reliability and interpretability. Artificial intelligence has the potential to bring driving changes to disease diagnosis methods through rapid traversal of medical images and efficient classification. However, the application of artificial intelligence in the field of medical image still faces challenges. Our method combines the multiple modalities of attention which consider the most discriminative part in the images. The proposed classification method is tested on the microscopic image dataset with 40 leukocyte categories, which achieves top-1 accuracy of 84.21% and top-5 accuracy of 99.44% during the testing procedure. And experiments on the dermoscopic image dataset show that our method has good generalization ability across multiple imaging modalities.
BadrinarayananV., KendallA. and CipollaR., Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence (2015).
3.
BikhetS.F., DarwishA.M., TolbaH.A. and ShaheenS.I., Segmentation and classification of white blood cells, In IEEE International Conference on Acoustics, 2000.
4.
ChenL.C., YiY., JiangW., WeiX. and YuilleA.L., Attention to scale: Scale-aware semantic image segmentation, In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
5.
ChunyingP., JikuiL. and LixiH., White blood cells image classification based on improving the connection of FCM and LFP, J Imag Graph18(5) (2013), 545–551.
6.
CodellaN.C.F., GutmanD., CelebiM.E., HelbaB., MarchettiM.A., DuszaS.W., KallooA., LiopyrisK., MishraN., KittlerH., et al., Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In Biomedical Imaging (ISBI 2018), 2018 IEEE 15th International Symposium on, IEEE, 2018, pp. 168–172.
7.
DengJ., DongW., SocherR., LiL.-J., LiK. and Fei-FeiL., ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255.
8.
GirshickR., DonahueJ., DarrellT. and MalikJ., Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 580–587.
HeK., ZhangX., RenS. and SunJ., Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
11.
HuJ., ShenL. and SunG., Squeeze-and-excitation networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7132–7141.
12.
HuangD.C., HungK.D. and ChanY.K., A computer assisted method for leukocyte nucleus segmentation and recognition in blood smear images, Journal of Systems & Software85(9) (2012), 2104–2118.
13.
HuangG., LiuZ., MaatenL.V.D. and WeinbergerK.Q., Densely connected convolutional networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4700–4708.
14.
JiangK., LiaoQ. and XiongY., A novel white blood cell segmentation scheme based on feature space clustering, Soft Computing10(1) (2006), 12–19.
15.
JingW., ZhilinZ., RaoB.D., ShiaofenF., JingwenY., SaykinA.J. and LiS., Identifying the neuroanatomical basis of cognitive impairment in Alzheimer’s disease by correlation- and nonlinearity-aware sparse Bayesian learning, IEEE Transactions on Medical Imaging33(7) (2014), 1475–87.
16.
KrizhevskyA., SutskeverI. and HintonG.E., ImageNet classification with deep convolutional neural networks, In International Conference on Neural Information Processing Systems, 2012.
17.
LabatiR.D., PiuriV. and ScottiF., ALL-IDB: The acute lymphoblastic leukemia image database for image processing, 2011, pp. 2045–2048.
18.
LeCunY., BengioY. and HintonG., Deep learning, Nature521(7553) (2015), 436.
19.
LinM., ChenQ. and ShuichengY., Network in network, International Conference on Learning Representations, 2013.
20.
LongJ., ShelhamerE. and DarrellT., Fully convolutional networks for semantic segmentation, IEEE Transactions on Pattern Analysis & Machine Intelligence39(4) (2014), 640–651.
21.
LongJ., ShelhamerE. and DarrellT., Fully convolutional networks for semantic segmentation, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 3431–3440.
22.
LoshchilovI. and HutterF., SGDR: Stochastic gradient descent with warm restarts, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
23.
MadhukarM., AgaianS.S. and AnthonyT., Chronopoulos, New decision support tool for acute lymphoblastic leukemia classification, Proceedings of SPIE8295 (2012), 829518.
24.
NairV. and HintonG.E., Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, pp. 807–814.
25.
NazlibilekS., KaracorD., ErcanT., SazliM.H., KalenderO. and EgeY., Automatic segmentation, counting, size determination and classification of white blood cells, Measurement55 (2014), 58–65.
26.
NewellA., YangK. and DengJ., Stacked hourglass networks for human pose estimation, In European Conference on Computer Vision, Springer, 2016, pp. 483–499.
27.
NohH., HongS. and HanB., Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1520–1528.
28.
PanS.J., YangQ., et al., A survey on transfer learning, IEEE Transactions on Knowledge and Data Engineering22(10) (2010), 1345–1359.
29.
PaszkeA., GrossS., ChintalaS., ChananG., YangE., DeVitoZ., LinZ., DesmaisonA., AntigaL. and LererA., Automatic differentiation in PyTorch, 2017.
30.
PatelN. and MishraA., Automated leukaemia detection using microscopic images, Procedia Computer Science58 (2015), 635–642.
31.
PengY. and LuB.L., Discriminative extreme learning machine with supervised sparsity preserving for image classification, Neurocomputing261 (2017).
32.
QinF., GaoN., PengY., WuZ., ShenS. and GrudtsinA., Fine-grained leukocyte classification with deep residual learning for microscopic images, Computer Methods and Programs in Biomedicine162 (2018), 243–252.
33.
RawatJ., SinghA., BhadauriaH.S., VirmaniJ. and DevgunJ.S., Classification of acute lymphoblastic leukaemia using hybrid hierarchical classifiers, Multimedia Tools and Applications76(18) (2017), 19057–19085.
34.
RezatofighiS.H. and Soltanian-ZadehH., Automatic recognition of five types of white blood cells in peripheral blood, Computerized Medical Imaging and Graphics35(4) (2011), 333–343.
35.
RipleyB.D., Pattern recognition and neural networks, Cambridge University Press, 2007.
36.
RobitailleL., DurandA., GardnerM., GagneC., KoninckP.D. and LavoiecardinalF., Learning to become an expert: Deep networks applied to super-resolution microscopy, National Conference on Artificial Intelligence, 2018, pp. 7805–7810.
37.
SaraswatM. and AryaK.V., Automated microscopic image analysis for leukocytes identification: A survey, Micron65 (2014), 20–33.
38.
SelvarajuR.R., CogswellM., DasA., VedantamR., ParikhD. and BatraD., Grad- CAM: Visual explanations from deep networks via gradientbased localization, In Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 618–626.
39.
ShafiqueS. and TehsinS., Acute lymphoblastic leukemia detection and classification of its subtypes using pretrained deep convolutional neural networks, Technology in Cancer Research & Treatment17 (2018), 153303381880278.
40.
ShaoY., LiuY., YeX. and ZhangS., A machine learning based global simulation data mining approach for efficient design changes, Advances in Engineering Software124 (2018), 22–41.
41.
SimonyanK. and ZissermanA., Very deep convolutional networks for large-scale image recognition, International Conference on Learning Representations, 2015.
42.
SzegedyC., LiuW., JiaY., SermanetP., ReedS., AnguelovD., ErhanD., VanhouckeV. and RabinovichA., Going deeper with convolutions, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.
43.
SzegedyC., VanhouckeV., IoffeS., ShlensJ. and WojnaZ., Rethinking the Inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818–2826.
44.
ThrunS. and PrattL., Learning to learn. Springer Science & Business Media, 2012.
45.
TschandlP., RosendahlC. and KittlerH., The HAM10000 dataset: A large collection of multi-source dermatoscopic images of common pigmented skin lesions, Scientific Data5 (2018), 180161.
46.
UshizimaD.M., LorenaA.C. and de CarvalhoA.C.P.L.F., Support vector machines applied to white blood cell recognition, In International Conference on Hybrid Intelligent Systems, 2005.
47.
VincentI., KwonK., LeeS. and MoonK., Acute lymphoid leukemia classification using two-step neural network classifier, 2015, pp. 1–4.
48.
VinyalsO., BlundellC., LillicrapT. and WierstraD., et al., Matching networks for one shot learning, In Advances in Neural Information Processing Systems, 2016, pp. 3630–3638.
49.
XuK., BaJ., KirosR., ChoK., CourvilleA., SalakhutdinovR., ZemelR. and BengioY., Show, attend and tell: Neural image caption generation with visual attention, Computer Science (2015), 2048–2057.
50.
YosinskiJ., CluneJ., BengioY. and LipsonH., How transferable are features in deep neural networks, In Advances in Neural Information Processing Systems, 2014, pp. 3320–3328.
51.
YuC., WangJ., PengC., GaoC., YuG. and SangN., Learning a discriminative feature network for semantic segmentation, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1857–1866.
52.
ZeilerM.D. and FergusR., Visualizing and understanding convolutional networks, In European Conference on Computer Vision, Springer, 2014, pp. 818–833.
53.
ZhangD., LuZ., ZhouX. and HeF., Integrating feature selection and feature extraction methods with deep learning to predict clinical outcome of breast cancer, IEEE Access6 (2018), 28936–28944.
54.
ZhaoJ., ZhangM., ZhouZ., ChuJ. and CaoF., Automatic detection and classification of leukocytes using convolutional neural networks, Medical & Biological Engineering & Computing55(8) (2017), 1287–1301.
55.
ZhouY., HeF. and QiuY., Accelerating image convolution filtering algorithms on integrated CPU-GPU architectures, Journal of Electronic Imaging27(3) (2018), 033002.
