Concept detection for a collection of images is an important topic and has recently been an emerging area of studies. Facing an imbalanced dataset is a great challenge in concept detection that has not yet been adequately investigated. To cope with this challenge, this paper proposes an image concept detection system based on the Convolutional Neural Network (CNN) method. The proposed method consists of three stages. At the first stage, a new algorithm is proposed to enhance the batch sampling in the CNN. At the second stage, some augmentation methods are used to improve learning process in CNN and at the final stage, a new ensemble of balanced convolutional neural network is presented in order to detect the concepts of images. Using Caltech-101 image dataset, the experimental results demonstrate the effectiveness of the proposed framework for concept detection in imbalanced datasets.
ZhengY.Shi-YongN.Tat-SengC. and QiT., Probabilistic optimized ranking for multimedia semantic concept detection via RVM, in: Proceeding of the International Conference on Content-Based Image and Video Retrieval, 2008, pp. 161–168.
2.
JiangY.JunY.Chong-WahN. and HauptmannA.G., Representations of keypoint-based semantic concept detection: A comprehensive study, IEEE Transactions on Multimedia12(1) (2010), 42–53.
3.
ShengT.ZhengY.WangY. and ChuaT., Sparse ensemble learning for concept detection, IEEE Transactions on Multimedia14(1) (2012), 43–54.
4.
ShyuM.HaruechaiyasakC. and ChenS., Category cluster discovery from distributed www directories, Information Sciences155(3-4) (2003), 181–197.
5.
ShyuM.ChenS.ChenM.ZhangC. and SarinnapakornK., Image database retrieval utilizing affinity relationships, in: Proceedings of the 1st ACM International Workshop on Multimedia Databases, 2003, pp. 78–85.
6.
LinL.ChenC.ShyuM. and ChenS., Weighted subspace filtering and ranking algorithms for video concept retrieval, IEEE Multi Media18(3) (2011), 32–43.
7.
TaoM.LiuY.ShyuM.YanY. and ShuC., Enhancing multimedia semantic concept mining and retrieval by incorporating negative correlations, in: IEEE International Conference on Semantic Computing (ICSC), 2014, pp. 28–35.
8.
YilinY.ChenM.ShyuM. and ChenS., Deep learning for imbalanced multimedia data classification, in: IEEE International Symposium on Multimedia (ISM), 2015, pp. 483–488.
9.
WillZ.ZhuS.YuK. and NgA.Y., Deep learning of invariant features via simulated fixations in video, in: Advances in Neural Information Processing Systems, 2012, pp. 3203–3211.
10.
YangY. and ShahM., Complex events detection using data-driven concepts, in: European Conference on Computer Vision, 2012, pp. 722–735.
11.
GangS. and DaiQ., A novel double deep ELMs ensemble system for time series forecasting, Knowledge-Based Systems134 (2017), 31–49.
12.
KrizhevskyA.Ilya SutskeverA. and HintonG.E., Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, 2012, pp. 1097–1105.
13.
LvY.DuanY.KangW.LiZ. and WangF.-Y., Traffic flow prediction with big data: A deep learning approach, IEEE Transaction on Intelligent Transportation Systems16(2) (2015), 865–873.
14.
TakashiK.KimuraS.KobayashiK. and ObayashiM., Time series forecasting using a deep belief network with restricted Boltzmann machines, Neurocomputing137 (2014), 47–56.
15.
ShengT.ZhangY.XuZ.LiH.ZhengY. and LiJ., An efficient concept detection system via sparse ensemble learning, Neurocomputing169 (2015), 124–133.
16.
JunY.YanR. and HauptmannA.G., Cross-domain video concept detection using adaptive svms, in: Proceedings of the 15th ACM International Conference on Multimedia, 2007, pp. 188–197.
17.
LiZ.ShiZ.ZhaoW.LiZ. and TangZ., Learning semantic concepts from image database with hybrid generative/discriminative approach, Engineering Applications of Artificial Intelligence26(9) (2013), 2143–2152.
18.
ZarchiM.TanR.T.Van GemerenC.MonadjemiA. and VeltkampR.C., Understanding image concepts using ISTOP model, Pattern Recognition53 (2016), 174–183.
19.
HaH.YangY.PouyanfarS.TianH. and ChenS., Correlation-based deep learning for multimedia semantic concept detection, in: International Conference on Web Information Systems Engineering, 2015, pp. 473–487.
20.
YilinY.ChenM.SadiqS. and ShyuM., Efficient imbalanced multimedia concept retrieval by deep learning on spark clusters, International Journal of Multimedia Data Engineering and Management (IJMDEM)8(1) (2017), 1–20.
21.
HuiskesM.J.ThomeeB. and LewM.S., New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative, in: Proceedings of the International Conference on Multimedia Information Retrieval, 2010, pp. 527–536.
22.
SunY.SudoK. and TaniguchiY., Visual concept detection of web images based on group sparse ensemble learning, Multimedia Tools and Applications75(3) (2016), 1409–1425.
23.
GuoY.LiuY.OerlemansA.LaoS.WuS. and LewM.S., Deep learning for visual understanding: A review, Neurocomputing187 (2016), 27–48.
24.
MobahiH.RonanC. and WestonJ., Deep learning from temporal coherence in video, in: Proceedings of the 26th Annual International Conference on Machine Learning, 2009, pp. 737–744.
25.
ZhangJ.LiS. and YinZ., Pattern classification of instantaneous mental workload using ensemble of convolutional neural networks, IFAC-PapersOnLine50(1) (2017), 14896–14901.
26.
ZhengZ.LiZ.NagarA. and KangW., Compact deep convolutional neural networks for image classification, ICMEW (2015), 1–6.
27.
YannL.BottouL.BengioY. and HaffnerP., Gradient-based learning applied to document recognition, Proceedings of the IEEE Conference11 (1998), 2278–2324.
28.
HuangF.J. and LeCunY., Large-scale learning with svm and convolutional for generic object categorization, Computer Vision and Pattern Recognition, IEEE Computer Society Conference1 (2006), 284–291.
29.
SerreT.LiorW.BileschiS.RiesenhuberM. and PoggioT., Robust object recognition with cortex-like mechanisms, IEEE Transactions on Pattern Analysis & Machine Intelligence3 (2007), 411–426.
30.
LiF.F.FergusR. and PeronaP., One-shot learning of object categories, IEEE Transactions on Pattern Analysis and Machine Intelligence28(4) (2006), 594–611.
31.
RameshB.XiangC. and LeeT.H., Multiple object cues for high performance vector quantization, Pattern Recognition67 (2017), 380–395.
32.
WangX.FengB.BaiX.LiuW. and Jan LateckiL., Bag of contour fragments for robust shape classification, Pattern Recognition47(6) (2014), 2116–2125.
33.
MuQ. and LiJ., Distance-based mixture modeling for classification via hypothetical local mapping, Statistical Analysis and Data Mining: The ASA Data Science Journal9(1) (2016), 43–57.
34.
LinG.FanC.ZhuH.MiuY. and KangX., Visual feature coding based on heterogeneous structure fusion for image classification, Information Fusion36 (2017), 275–283.
35.
VahdatpourM.SajediH. and RamezaniF., Air pollution forecasting from sky images with shallow and deep classifiers, Earth Science Informatics11(3) (2018), 413–422.
