During the last decade, a variety of ensembles methods has been developed. All known and widely used methods of this category produce and combine different learners utilizing the same algorithm as the basic classifiers. In the present study, we use two well-known approaches, namely, Rotation Forest and Random Subspace, in order to increase the effectiveness of a single learning algorithm. We have conducted experiments with other well-known ensemble methods, with 25 sub-classifiers, in order to test the proposed model. The experimental study that we have conducted is based on 35 various datasets. According to the Friedman test, the Rotation Forest of Random Subspace C4.5 (RFRS C4.5) and the PART (RFRS PART) algorithms exhibit the best scores in our resulting ranking. Our results have shown that the proposed method exhibits competitive performance and better accuracy in most of the cases.
WolpertDHMacreadyWG. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation.1997; 1(1): 67-82.
2.
AdamSPAlexandropoulosSANPardalosPMVrahatisMN. No free lunch theorem: A review. Approximation and Optimization.2019; pp. 57-82.
3.
PardalosPMRasskazovaVVrahatisMN, editors. Black Box Optimization, Machine Learning, and No-Free Lunch Theorems. vol. 170 Springer Optimization and Its Applications-Mathematics. Springer, 2021.
4.
DietterichTG. Ensemble methods in machine learning. In: International Workshop on Multiple Classifier Systems. Springer, 2000; pp. 1-15.
5.
OpitzDMaclinR. Popular ensemble methods: An empirical study. Journal of Artificial Intelligence Research.1999; 11: 169-198.
6.
AličkovićESubasiA. Breast cancer diagnosis using GA feature selection and Rotation Forest. Neural Computing and Applications.2017; 28(4): 753-763.
7.
ZhuHJYouZHZhuZXShiWLChenXChengL. DroidDet: effective and robust detection of android malware using static analysis along with rotation forest model. Neurocomputing.2018; 272: 638-646.
8.
RodriguezJJKunchevaLIAlonsoCJ. Rotation forest: A new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence.2006; 28(10): 1619-1630.
9.
HoTK. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence.1998; 20(8): 832-844.
FreundYSchapireRE. Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning. vol. 96. Morgan Kaufmann; 1996. pp. 148-156.
12.
MelvillePMooneyRJ. Constructing diverse classifier ensembles using artificial training examples. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence. vol. 3. Morgan Kaufmann; 2003. pp. 505-510.
13.
ZhouZH. Ensemble methods: foundations and algorithms. CRC press, 2012.
14.
RokachLMaimonO. Decision trees. In: Data Mining and Knowledge Discovery Handbook. Springer; 2005; pp. 165-192.
15.
CohenWWSingerY. A simple, fast, and effective rule learner. AAAI/IAAI.1999; 99(335-342): 3.
16.
BunkhumpornpatCSinapiromsaranKLursinsapC. Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer; 2009; pp. 475-482.
17.
RosFGuillaumeS. Sampling Techniques for Supervised or Unsupervised Tasks. Springer, 2020.
18.
YariyanPJanizadehSVan PhongTNguyenHDCostacheRVan LeH, et al. Improvement of best first decision trees using bagging and dagging ensembles for flood probability mapping. Water Resources Management.2020; 34(9): 3037-3053.
WebbGI. Multiboosting: A technique for combining boosting and wagging. Machine Learning.2000; 40(2): 159-196.
21.
KunchevaLISkurichinaMDuinRP. An experimental study on diversity for bagging and boosting with linear classifiers. Information Fusion.2002; 3(4): 245-258.
22.
BreimanL. Random forests. Machine Learning.2001; 45(1): 5-32.
23.
VidalRMaYSastrySS. Generalized Principal Component Analysis. Springer;2016.
24.
PhamBTBuiDTPrakashIDholakiaM. Rotation forest fuzzy rule-based classifier ensemble for spatial prediction of landslides using GIS. Natural Hazards.2016; 83(1): 97-127.
25.
BurroughPAMcDonnellRMcDonnellRALloydCD. Principles of Geographical Information Systems. Oxford University Press, 2015.
26.
BlaserRFryzlewiczP. Random rotation ensembles. The Journal of Machine Learning Research.2016; 17(1): 126-151.
27.
DuaDGraffC. UCI Machine Learning Repository, University of California, School of Information and Computer Science, Irvine, CA, 2019; 2019.
28.
WittenIFrankEHallMPalC. Data mining fourth edition: Practical machine learning tools and techniques. San Francisco: Morgan Kaufmann Publishers Inc;2016.
QuinlanJR. Induction of decision trees. Machine Learning.1986; 1(1): 81-106.
31.
RoelofsRShankarVRechtBFridovich-KeilSHardtMMillerJ, et al. A meta-analysis of overfitting in machine learning. Advances in Neural Information Processing Systems.2019; 32: 9179-9189.
32.
AlexandropoulosSANKotsiantisSBVrahatisMN. Data preprocessing in predictive data mining. The Knowledge Engineering Review.2019; 34.
33.
RuggieriS. Efficient C4.5 [classification algorithm]. IEEE Transactions on Knowledge and Data Engineering. 2002; 14(2): 438-444.
34.
HodgesJLehmannEL. Rank methods for combination of independent experiments in analysis of variance. In: Selected Works of EL Lehmann. Springer;2012; pp. 403-418.
35.
FinnerH. On a monotonicity problem in step-down multiple test procedures. Journal of the American Statistical Association.1993; 88(423): 920-923.
36.
WeissSMIndurkhyaN. Rule-based machine learning methods for functional prediction. Journal of Artificial Intelligence Research.1995; 3: 383-403.
37.
FrankEWittenIH. Generating accurate rule sets without global optimization. 1998.
38.
DžeroskiSŽenkoB. Is combining classifiers with stacking better than selecting the best one? Machine Learning.2004; 54(3): 255-273.
39.
MurtyMNDeviVS. Combination of classifiers. In: Pattern Recognition. Springer;2011; pp. 188-206.
