Since label noise can hurt the performance of supervised learning (SL), how to train a good classifier to deal with label noise is an emerging and meaningful topic in machine learning field. Although many related methods have been proposed and achieved promising performance, they have the following drawbacks: (1) They can lead to data waste and even performance degradation if the mislabeled instances are removed; and (2) the negative effect of the extremely mislabeled instances cannot be completely eliminated. To address these problems, we propose a novel method based on the capped ℓ1 norm and a graph-based regularizer to deal with label noise. In the proposed algorithm, we utilize the capped ℓ1 norm instead of the ℓ1 norm. The used norm can inherit the advantage of the ℓ1 norm, which is robust to label noise to some extent. Moreover, the capped ℓ1 norm can adaptively find extremely mislabeled instances and eliminate the corresponding negative influence. Additionally, the proposed algorithm makes full use of the mislabeled instances under the graph-based framework. It can avoid wasting collected instance information. The solution of our algorithm can be achieved through an iterative optimization approach. We report the experimental results on several UCI datasets that include both binary and multi-class problems. The results verified the effectiveness of the proposed algorithm in comparison to existing state-of-the-art classification methods.
AharonM., EladM. and BrucksteinA., K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing54 (2006), 4311–4322.
2.
AllahbakhshM., BenatallahB., IgnjatovicA., Motahari-NezhadH.R., BertinoE. and DustdarS., Quality control in crowdsourcing systems: Issues and directions, IEEE Internet Computing17(2) (2013), 76–81.
3.
BelkinM., NiyogiP. and SindhwaniV., Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, Journal of Machine Learning Research7 (2006), 2399–2434.
4.
BhadraS. and HeinM., Correction of noisy labels via mutual consistency check, Neurocomputing160 (2015), 34–52.
5.
CaoY., HeH. and HuangH., Lift: A new framework of learning from testing data for face recognition, Neurocomputing74(6) (2011), 916–929.
6.
BrodleyC.E., M.A.F., Identifying and eliminating mislabeled training instances. In: Proceedings of the National Conference on Artificial Intelligence, The MIT Press, Oregon, USA, (1996), pp. 799–805.
7.
CaruanaR., Niculescu-MizilA., An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd International Conference on Machine Learning, ACM, NY, USA, (2006), pp. 161–168.
8.
ChenM., WangQ., ChenS. and LiX., Capped l1-norm sparse representation method for graph clustering, IEEE Access7 (2019), 54464–54471.
9.
ChenW.-J., ShaoY.-H., LiC.-N. and DengN.-Y., MLTSVM: A novel twin support vector machine to multi-label learning, Pattern Recognition52 (2016), 61–74.
10.
ChengG., LiZ., HanJ., YaoX. and GuoL., Exploring hierarchical convolutional features for hyperspectral image classification, IEEE Transactions on Geoscience and Remote Sensing56(11) (2018), 6712–6722.
11.
ChhogyalK., NayakA., An empirical study of a simple naive bayes classifier based on ranking functions, In: Proceedings of Australasian Joint Conference on Artificial Intelligence, Springer International Publishing, Cham, (2016), pp. 324–331.
12.
CollinsM., SchapireR.E. and SingerY., Logistic regression, adaboost and bregman distances, Machine Learning48(1) (2002), 253–285.
13.
FoodyG.M., The effect of mis-labeled training data on the accuracy of supervised image classification by svm, In: Proceedings of the 2015 IEEE International Geoscience and Remote Sensing Symposium, IEEE, Milan, Italy, (2015), pp. 4987–4990.
FrenayB. and VerleysenM., Classification in the presence of label noise: A survey, IEEE Transactions on Neural Networks and Learning Systems25(5) (2014), 845–869.
16.
GanH., HuangR., LuoZ., XiX. and GaoY., On using supervised clustering analysis to improve classification performance, Information Sciences454-455 (2018), 216–228.
17.
GanH., SangN., ChenX., Semi-supervised kernel minimum squared error based on manifold structure, In: Proceedings of the 10th International Symposium on Neural Networks, Springer-Verlag, Dalian, China, (2013), pp. 265–272.
18.
GanH., SheQ., MaY., WuW. and MengM., Generalization improvement for regularized least squares classification, Neural Computing and Applications31(2) (2019), 1045–1051.
19.
GhoshA., KumarH., SastryP.S., Robust loss functions under label noise for deep neural networks. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI Publications, San Francisco, California, USA, (2017), pp. 1919–1925.
20.
GhoshA., ManwaniN. and SastryP., Making risk minimization tolerant to label noise, Neurocomputing160 (2015), 93–107.
21.
GongC., ZhangH., YangJ., TaoD., Learning with inadequate and incorrect supervision. In: Proceedings of the 2017 IEEE International Conference on Data Mining. IEEE, Los Alamitos, CA, USA, (2017), pp. 889–894.
22.
IpeirotisP.G., ProvostF., WangJ., Quality management on amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, ACM, New York, NY, USA, (2010), pp. 64–67.
23.
JiangW., NieF., HuangH., Robust dictionary learning with capped l1-norm. In: Proceedings of International Joint Conference on Artificial Intelligence, AAAI Press, Buenos Aires, Argentina, (2015), pp. 3590–3596.
24.
KangX., DuanP., XiangX., LiS. and BenediktssonJ.A., Detection and correction of mislabeled training samples for hyperspectral image classification, IEEE Transactions on Geoscience and Remote Sensing56(10) (2018), 5673–5686.
25.
KearnsM., Efficient noise-tolerant learning from statistical queries, Journal of the ACM45(6) (1998), 983–1006.
26.
LeCunY., BengioY. and HintonG., Deep learning, Nature521(7553) (2015), 436–444.
27.
LiR., WangX., LeiL. and SongY., l21 -norm based loss function and regularization extreme learning machine, IEEE Access7 (2019), 6575–6586.
28.
LiX., ShenH., ZhangL., ZhangH., YuanQ. and YangG., Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning, IEEE Transactions on Geoscience and Remote Sensing52 (2014), 7086–7098.
29.
LiuQ. and LiuC., A novel locally linear knn method with applications to visual recognition, IEEE Transactions on Neural Networks and Learning Systems28(9) (2016), 2010–2021.
30.
LiuT. and TaoD., Classification with noisy labels by importance reweighting, IEEE Transactions on Pattern Analysis and Machine Intelligence38(3) (2016), 447–461.
31.
LiuZ., XueS., ZhangL., PuJ. and WangH., An improved kernel minimum square error classification algorithm based on l2,1-norm regularization, IEEE Access5 (2017), 14133–14140.
32.
ManwaniN. and SastryP.S., Noise tolerance under risk minimization, IEEE Transactions on Cybernetics43(3) (2013), 1146–1151.
33.
MaoK., CapraL., HarmanM. and JiaY., A survey of the use of crowdsourcing in software engineering, Journal of Systems and Software126 (2017), 57–84.
34.
MuhlenbachF., LallichS. and ZighedD.A., Identifying and handling mislabelled instances, Journal of Intelligent Information Systems22(1) (2004), 89–109.
35.
NatarajanN., DhillonI.S., RavikumarP.K., TewariA., Learning with noisy labels, In: Advances in Neural Information Processing Systems 26. Curran Associates, Inc., Lake Tahoe, Nevada, USA, (2013), pp. 1196–1204.
36.
NettletonD.F., Orriols-PuigA. and FornellsA., A study of the effect of different types of noise on the precision of supervised learning techniques, Artificial Intelligence Review33(4) (2010), 275–306.
37.
NieF., WangX., HuangH., Multiclass capped lp-norm svm for robust classifications. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI Press, San Francisco, California, USA, (2017), pp. 2415–2421.
38.
RätschG., OnodaT. and MüllerK.-R., Soft margins for adaboost, Machine Learning42(3) (2001), 287–320.
39.
RifkinR., YeoG. and PoggioT., Regularized least-squares classification, Nato Science Series Sub Series III Computer and Systems Sciences190 (2003), 131–154.
40.
RodriguezA. and LaioA., Clustering by fast search and find of density peaks, Science344(6191) (2014), 1492–1496.
41.
ScottC., BlanchardG., HandyG., Classification with asymmetric label noise: Consistency and maximal denoising, In: Proceedings of the 26th Annual Conference on Learning Theory, PMLR, Princeton, NJ, USA, (2013), pp. 489–511.
42.
SmithM.R., MartinezT., Improving classification accuracy by identifying and removing instances that should be misclassified, In: Proceedings of the 2011 International Joint Conference on Neural Networks, IEEE, San Jose, California, USA, (2011), pp. 2690–2697.
43.
TuB., ZhangX., KangX., ZhangG. and LiS., Density peak-based noisy label detection for hyperspectral image classification, IEEE Transactions on Geoscience and Remote Sensing57(3) (2019), 1573–1584.
VapnikV.N. and VapnikV., Statistical learning theory, Wiley, (1998).
46.
VaradarajanB., YuD., DengL., AceroA., Using collective information in semi-supervised learning for speech recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. IEEE, Taipei, Taiwan, (2009), pp. 4633–4636.
47.
WangA., LuJ., CaiJ., ChamT. and WangG., Large-margin multi-modal deep learning for RGB-D object recognition, IEEE Transactions on Multimedia17(11) (2015), 1887–1898.
48.
XiaoT., XiaT., YangY., HuangC., WangX., Learning from massive noisy labeled data for image classification. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, MA, USA, (2015), pp. 2691–2699.
ZhangJ., ShengV.S., WuJ., FuX., WuX., Improving label quality in crowdsourcing using noise correction. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, ACM, New York, NY, USA, (2015), pp. 1931–1934.
51.
ZhangL. and ZhangD., Robust visual knowledge transfer via extreme learning machine-based domain adaptation, IEEE Transactions on Image Processing25(10) (2016), 4959–4973.
52.
ZhangL., ZuoW. and ZhangD., LSDT: Latent sparse domain transfer learning for visual adaptation, IEEE Transactions on Image Processing25(3) (2016), 1177–1191.
53.
ZhuX., LiuY., LiJ., WanT. and QinZ., Emotion classification with data augmentation using generative adversarial networks. In: Advances in Knowledge Discovery and Data Mining, Springer International Publishing, Cham, (2018), pp. 349–360.
