iBoost: Boosting using an instance-based exponential weighting scheme

Abstract

AdaBoost is a well-recognized ensemble method to improve prediction accuracy over the base learning algorithm. However, it is prone to overfitting the training instances [18]. Freund, Mansour and Schapire [5] established that using exponential weighting scheme in combining classifiers reduces the problem of overfitting. Also, Helmbold, Kwek and Pitt [7] showed in the prediction using a pool of experts framework an instance-based weighting scheme improves performance. Motivated by these results, we propose here an instance-based exponential weighting scheme in which the weights of the base classifiers are adjusted according to the test instance x. Here, a competency classifier ${\mathit{c}}_{\mathit{i}}$ is constructed for each base classifier ${\mathit{h}}_{\mathit{i}}$ to predict whether the base classifier’s guess of x’s label can be trusted and adjust the weight of ${\mathit{h}}_{\mathit{i}}$ accordingly. We show that this instance-based exponential weighting scheme enhances the performance of AdaBoost.