Sage Journals: Discover world-class research

Abstract

Empirical evidence suggests that ensembles with adequate levels of pairwise diversity among a set of accurate member algorithms can significantly outperform any of the individual algorithms. As a result, several diversity measures have been developed for use in optimizing ensembles. We show, however, that there is natural tension between the pairwise diversity of ensemble members and their individual accuracy. While efficient ensembles can be built with stronger forms of diversity, they also suffer in overall accuracy. On the other hand, ensembles built with weaker forms of diversity can be very accurate, but tend to be significantly more computationally expensive. We discuss these findings in light of the notion of diversity space.

Keywords

Diversity ensemble learning metalearning

Get full access to this article

View all access options for this article.

References

Breiman

, Random forests, Machine Learning 45(1) (2001), 5–32.

Dietterich

T.G.

, Ensemble Methods in Machine Learning, in: Proceedings of the First International Workshop on Multiple Classifier Systems, Springer, Berlin, Heidelberg, 2000, pp. 1–15.

Eibe

Hall

M.A.

and Witten

I.H.

, The WEKA Workbench, Online Appendix for Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann, Fourth Edition, 2016.

Gatnar

, A diversity measure for tree-based classifier ensembles, in: Data Analysis and Decision Support. Baier

Decker

and Schmidt-Thieme

, eds., Springer, Berlin, Heidelberg, 2005, pp. 30–38.

Giacinto

and Roli

, Design of effective neural network ensembles for image classification purposes, Image and Vision Computing 19(9) (2001), 699–707.

Kuncheva

L.I.

and Whitaker

C.J.

, Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy, Machine Learning 51(2) (2003), 181–207.

Lee

J.W.

and Giraud-Carrier

, A metric for unsupervised metalearning, Intelligent Data Analysis 15(6) (2011), 827–841.

Peterson

A.H.

and Martinez

T.R

, Estimating the potential for combining learning models, in: Proceedings of the ICML Workshop on Meta-learning 2005, pp. 68–75.

R Development Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2007.

10.

Rudolph

and Martinez

, Finding the real differences between learning algorithms, International Journal on Artificial Intelligence Tools 24(3) (2015), 1550001.

11.

Skalak

D.B.

, The sources of increased accuracy for two proposed boosting algorithms, in: Proceedings of the AAAI Workshop on Integrating Multiple Learned Models Workshop, 1996, pp. 120–125.

12.

Tang

E.K.

Suganthan

P.N.

and Yao

, An analysis of diversity measures, Machine Learning 65(1) (2006), 247–271.

13.

Wolpert

D.H.

, Stacked generalization, Neural Networks 5(2) (1992), 241–259.

14.

Yule

G.U.

, VII. On the association of attributes in statistics: with illustrations from the material of the childhood society, &c., Philosophical Transactions of the Royal Society A 194(252–261) (1992), 257–319.

15.

Zeng

Wong

D.F.

and Chao

L.S.

, Constructing better classifier ensemble based on weighted accuracy and diversity measure, The Scientific World Journal, 2014.

Diversity,accuracy and efficiency in ensemble learning: An unexpected result

Abstract

Keywords

Get full access to this article

References