Regression via Classification (RvC) is a process to solve a regression problem by using a classifier. An ensemble consists of many models, in which the final result is the combination of the results of these individual models. In this paper, two RvC ensemble methods are proposed. In the first ensemble method, the output of the ensemble method is modified to achieve the final output. A formula is derived in this paper for this purpose. In the second method, a new approach is proposed to compute the output of each model of an ensemble. It is shown that an accurate binary classifier can be transformed into an accurate regression method with the proposed methods. It is also shown experimentally, by using popular Random Forests as a classifier in the proposed ensemble methods against Random Forests as a regression method, the effectiveness of the proposed RvC ensemble methods.
AhmadA., Data Transformation for Decision Tree Ensembles, PhD thesis, School of Computer Science, University of Manchester2010.
2.
AhmadA., HalawaniS.M. and AlbidewiI., Novel ensemble methods for regression via classification problems, Expert Systems with Applications39 (7) (2012), 6396–6401.
3.
AhmadA., HalawaniS.M. and AlbidewiI., Consistency of randomized and finite sized decision tree ensembles, Pattern Analysis And Applications17 (1) (2014), 97–104.
4.
AlpaydinE., Combined 5×2 cv f test comparing supervised classification learning algorithms, Neural Computation11 (8) (1999), 1885–1892.
5.
BishopC.M., Pattern Recognition and Machine Learning. Springer-Verlag New York Inc, 2008.
6.
BoslaughS., Statistics in a Nutshell: A Desktop Quick Reference. O’Reilly Media, 2012.
7.
BreimanL., Some Infinite Theory for Predictor Ensembles. Technical Report 577, Statistics Department, UC Berkeley, 2000.
8.
BreimanL., Random forests, Machine Learning45 (1) (2001), 5–32.
9.
BreimanL., FriedmanJ., OlshenR., StoneC., Classification and Regression Trees, Wadsworth International Group, CA, 1984.
10.
DaiQ., A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot, Neurocomputing122 (2013), 258–265.
11.
DaiQ. and LiM., Introducing randomness into greedy ensemble pruning algorithms, Applied Intelligence42 (2015), 406–429.
12.
DietterichT.G., Ensemble methods in machine learning, In Proc of Conf Multiple Classifier Systems, 1857, 2000, pp. 1–15.
13.
DietterichT.G. and BakiriG., Solving multiclass learning problems via error-correcting output codes, J Artificial Intelligence Research2 (1995), 263–286.
14.
DoughertyJ., KahaviR. and SahamiM., Supervised and unsupervised discretization of continuous features, Proceedings of the Twelth International Conference, In In Machine Learning, 1995.
15.
FrankE.E., BouckaertR., Conditional density estimation with class probability estimators, In Advances in Machine Learning, Berlin Heidelberg, Springer Heidelberg, 2009, pp. 65–81.
16.
FanW., McCloskeyJ. and YuP.S., A General Framework for Accurate and Fast Regression by Data Summarization in Random Decision Trees, In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006, pp. 136–146.
17.
FrankA. and AsuncionA., UCI machine learning repository, 2010.
18.
FreundY. and SchapireR.E., A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences55 (1) (1997), 119–139.
19.
FumeraG., RoliF. and SerrauA., A theoretical analysis of bagging as a linear combination of classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence30 (7) (2008), 1293–1299.
20.
HalawaniS.M., AlbidewiI.A. and AhmadA., A novel ensemble method for regression via classification problems, Journal of Computer Science7 (3) (2011), 387–393.
21.
HallM., FrankE., HolmesG., PfahringerB., ReutemannP. and WittenI.H., The WEKA data mining software: An update, SIGKDD Explorations11 (1) (2009), 10–18.
22.
IndurkhyaN. and WeissS.M., Solving Regression Problems with Rule-based Ensemble Classifiers, In ACM International Conference Knowledge Discovery and Data Mining (KDD01), 2001, pp. 287–292.
23.
JanssenF. and FurnkranzJ., Heuristic Rule-based Regression via Dynamic Reduction to Classification, In Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence. 2011, pp. 1330–1335.
24.
KrawczykB., WozniakM. and CyganekB., Clustering-based ensembles for one-class classification, Information Sciences264 (2014), 182–195.
25.
KunchevaL.I., Combining Pattern Classifiers: Methods and Algorithms, Wiley-Interscience, 2004.
26.
LangfordJ. and ZadroznyB., Estimating Class Membership Probabilities using Classifier Learners, In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, 2005, pp. 198–205.
PartalasI., TsoumakasG. and VlahavasI., Focused Ensemble Selection: A Diversity-based Method for Greedy Enble Selection, Proceedings of the European Conference on Artificial Intelligence, 2008, pp. 117–121.
29.
PolikarR., Ensemble Based Systems in Decision Making, IEEE Circuits and Systems Magazine, Third Quarter, 2006, 21–45.
30.
QuinlanJ.R., C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993.
31.
R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2008, ISBN 3-900051-07-0.
32.
TorgoL. and GamaJ., Regression by Classification, In Advances in Artificial Intelligence, 1996, pp. 51–60.
33.
TorgoL. and GamaJ., Search-based Class Discretization, In Proceedings of the 9th European Conference on Machine Learning, 1997, PP. 266–273.
34.
ZainalA., MaarofM.A., HjS.M., Shamsuddin and AbrahamA.. Ensemble of one-class classifiers for network intrusion detection system, In Proccedings of the Fourth International Conference on Information Assurance and Security, IAS 2008, Napoli, Italy, 2008.
