Breakthrough classification performances have been achieved by utilizing ensemble techniques in machine learning and data mining. Bagging is one such ensemble technique that has outperformed single models in obtaining higher predictive performances. This paper proposes an ensemble technique by utilizing the basic bootstrap aggregating technique on hybridization of two base learners namely Naïve Bayes (NB) and Decision Tree (DT). Before induction of the DT, NB algorithm is employed for eliminating mislabeled or contradictory instances from the training set. Consequently, bagging approach is applied on hybrid NBDT as the base learner. The resultant Bagged Naïve Bayes-Decision Tree (BNBDT) algorithm is then used for improving the classification accuracy of various multi-class problems. This algorithm iteratively trains the base learner from random samples of the training set, and then performs majority voting of their predictions. The proposed algorithm is compared with both ensemble and single classification techniques such as Random Forest, Bagged NB, Bagged DT, NB, and DT. Experimental results over 52 UCI data sets with bag size 100 demonstrate that the proposed algorithm significantly outperforms the existing algorithms.
I.Kavakiotis, O.Tsave, A.Salifoglou, N.Maglaveras, I.Vlahavas and I.Chouvarda, Machine learning and data mining methods in diabetes research, Comput Struct Biotechnol J15 (2017), 104–116.
2.
S.H.Liao, P.H.Chu and P.Y.Hsiao, Data mining techniques and applications - A decade review from 2000 to 2011, Expert Syst Appl39 (2012), 11303–11311.
3.
P.Langley and S.Sage, Induction of Selective Bayesian Classifiers, In: Uncertainty Proceedings1994, 1994, pp. 399–406. Elsevier.
4.
Z.-H.Zhou, Ensemble Learning, In: Encyclopedia of Biometrics, Springer US, Boston, MA, 2009, pp. 270–273.
5.
A.Rocha and S.K.Goldenstein, Multiclass from binary: Expanding One-versus-all, one-versus-one and ECOC-based approaches, IEEE Trans Neural Networks Learn Syst25 (2014), 289–302.
6.
A.Chaudhary, S.Kolhe and R.Kamal, A hybrid ensemble for classification in multiclass datasets: An application to oilseed disease dataset, Comput Electron Agric124 (2016), 65–72.
7.
M.Aly, Survey on multiclass classification methods, Neural Netw19 (2005), 1–9.
8.
D.Silva-Palacios, C.Ferri and M.J.Ramírez-Quintana, Probabilistic class hierarchies for multiclass classification, J Comput Sci26 (2018), 254–263.
9.
A.Chaudhary, S.Kolhe and R.Kamal, An improved random forest classifier for multi-class classification, Inf Process Agric3 (2016), 215–222.
10.
N.García-Pedrajas and D.Ortiz-Boyer, An empirical study of binary classifier fusion methods for multiclass classification, Inf Fusion12 (2011), 111–130.
11.
R.Mousavi and M.Eftekhari, A new ensemble learning methodology based on hybridization of classifier ensemble selection approaches, Appl Soft Comput J37 (2015), 652–666.
12.
N.Agarwal, V.N.Balasubramanian and C.V.Jawahar, Improving multiclass classification by deep networks using DAGSVM and Triplet Loss, Pattern Recognit Lett112 (2018), 184–190.
13.
I.H.Sarker, M.A.Kabir, A.Colman and J.Han, An Improved Naive Bayes Classifier-Based Noise Detection Technique for Classifying User Phone Call Behavior, In: Australasian Conference on Data Mining, Springer, Singapore, 2018, pp. 72–85.
14.
S.Ren, Y.Lian and X.Zou, Incremental naïve bayesian learning algorithm based on classification contribution degree, J Comput9 (2014), 1967–1974.
15.
A.Frank and A.Asuncion, UCI Machine Learning Repository. University of California, School of Information and Computer Science, http://archive.ics.uci.edu/ml.
16.
A.De Caigny, K.Coussement and K.W.De Bock, A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees, Eur J Oper Res269 (2018), 760–772.
17.
S.Kotsiantis, A hybrid decision tree classifier, J Intell Fuzzy Syst26 (2014), 327–336.
18.
D.R.Carvalho and A.A.Freitas, A hybrid decision tree/genetic algorithm method for data mining, Inf Sci (Ny)163 (2004), 13–35.
19.
A.Panhalkar and D.Doye, An outlook in some aspects of hybrid decision tree classification approach: A survey, In: Proceedings of the International Conference on Data Engineering and Communication Technology, Springer, Singapore, 2017, pp. 85–95.
20.
L.-M.Wang, X.-L.Li, C.-H.Cao and S.-M.Yuan, Combining decision tree and Naive Bayes for classification, Knowledge-Based Syst19 (2006), 511–515.
21.
K.Polat and S.Güneş, A novel hybrid intelligent method based on C4. 5 decision tree classifier and one-against-all approach for multi-class classification problems, Expert Syst Appl36 (2009), 1587–1592.
22.
S.-J.Lee, Z.Xu, T.Li and Y.Yang, A novel bagging C4. 5 algorithm based on wrapper feature selection for supporting wise clinical decision making, J Biomed Inform78 (2017), 144–155.
23.
P.Singh and S.Verma, Multi-classifier model for software fault prediction, Int Arab J Inf Technol15 (2018), 912–919.
24.
M.Ala’raj and M.F.Abbod, Classifiers consensus system approach for credit scoring, Knowledge-Based Syst104 (2016), 89–105.
25.
J.Sun, J.Lang, H.Fujita and H.Li, Imbalanced enterprise credit evaluation with DTE-SBD: Decision tree ensemble based on SMOTE and bagging with differentiated sampling rates, Inf Sci (Ny)425 (2018), 76–91.
26.
J.Wu, S.Pan, X.Zhu, Z.Cai, P.Zhang and C.Zhang, Self-adaptive attribute weighting for Naive Bayes classification, Expert Syst Appl42 (2015), 1487–1502.
27.
B.Chandra and M.Gupta, Robust approach for estimating probabilities in Naïve–Bayes Classifier for gene expression data, Expert Syst Appl38 (2011), 1293–1298.
28.
Wei, Z.Sen, J.Y.Yang, Shen, H.Bin and D.J.Yu, A cascade random forests algorithm for predicting protein-protein interaction sites, IEEE Trans Nanobioscience14 (2015), 746–760.
29.
L.Lin, F.Wang, X.Xie and S.Zhong, Random forests-based extreme learning machine ensemble for multi-regime time series prediction, Expert Syst Appl83 (2017), 164–176.
30.
D.Silva-Palacios, C.Ferri and M.J.Ramírez-Quintana, Improving performance of multiclass classification by inducing class hierarchies, Procedia Comput Sci108 (2017), 1692–1701.
C.A.Kulikowski and S.M.Weiss, Computer systems that learn: classification and prediction methods from statistics neural nets, machine learning, and expert systems, ,Morgan Kaufmann Publishers, San Francisco, 1991.
