This paper gives a survey of the inverse boosting algorithm which employs an inverse error vector to produce new sample distributions. The experimental result shows that the inverse boosting can outperform normal boosting in some cases. Further experiments on several benchmark real-world data sets available from the UCI repository are taken to get a better insight into the inverse boosting algorithm, and then we find that the performance of the ensemble is determined by the performance of its base classifiers. This distinctive feature of inverse boosting indicates that traditional ways to improve the performance of an ensemble through enhancing the diversity between the base classifier are invalid here. Therefore, this paper proposes a method with pertinence to improve the performance of inverse boosting.
DietterichTG (2000) An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting and randomization. Machine Learning40: 139–57.
3.
FreundYSchapireRE (1996) Experiments with a new boosting algorithm. Machine Learning: Proceeding of the Thirteenth International Conference, 22 January, 148–146, Morgan Kaufmann.
4.
FreundYSchapireRE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Science55: 119–39.
5.
Garcia-PedrajasNOrtiz-BoyerD (2008) Boosting random subspace method. Neural Networks21: 1344–62.
6.
HansenLSalamonP (1990) Neural networks ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence12: 993–1001.
7.
KunchevaLIWhitakerCJ (2001) Ten measures of diversity in classifier ensembles: limits for two classifiers. IEE Workshop on Intelligent Sensor Processing, Birmingham, 10/1–10/6.
8.
KunchevaLIWhitakerCJ (2002) Using diversity with three variants of Boosting. Multiple Classifier Systems, Proceedings, pp. 81–90.
9.
KunchevaLISkurichinaMDuinRPW (2001) An experimental study on diversity for bagging and boosting with linear classifiers. Information Fusion3: 245–58.
10.
Martinez-MunozGSuarezA (2010) Out-of-bag estimation of the optimal sample size in bagging. Pattern Recognition43: 143–52.
11.
OptizDMaclinR (1999) Popular ensemble methods: an empirical study. Journal of Artificial Intelligence Research11: 169–98.
12.
OzaNC (2003) Boosting with averaged weight vectors. Computer Science2709: 15–24.
13.
RodriguezJJKunchevaLI (2006) Rotation forest: a new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence28: 1619–29.
14.
RumelhartDEHintonGEWilliamsRJ (1986) Learning representations by back-propagating errors. Nature323: 533–36.
15.
SchapireRE (1990) The strength of weak learnability. Machine Learning5: 197–227.
16.
ShippCAKunchevaLI (2002) Relationships between combination methods and measures of diversity in combining classifiers. Information Fusion2: 135–48.
17.
TrespV (2001) Committee machines. In HuYHwangH-N (eds) Handbook for Neural Network Signal Processing. Boca Raton, FL: CRC Press.
18.
Torres-SospedraJHernandez-EspinosaCFernandez-RedondoM (2007a) Designing a multilayer feedforward ensemble with the weighted conservative boosting algorithm. Proceedings of the International Joint Conference on Neural Networks, pp. 684–689.
19.
Torres-SospedraJHernandez-EspinosaCFernandez-RedondoM (2007b) Mixing Aveboost and Conserboost to improve Boosting methods. Proceedings of the International Joint Conference on Neural Networks, pp. 672–677.
20.
Torres-SospedraJHernandez-EspinosaCFernandez-RedondoM (2008) Researching on combining Boosting ensembles. Proceedings of the International Joint Conference on Neural Networks, pp. 193–204.
21.
TumerKGhoshJ (1996a) Analysis of decision boundaries in linearly combined neural classifiers. Pattern Recognition29: 341–48.
22.
TumerKGhoshJ (1996b) Error correlation and error reduction in ensemble classifiers. Connection Science8: 385–404.
ZhangC-XZhangJ-S (2008a) RotBoost: a technique for combining Rotation Forest and AdaBoost. Pattern Recognition Letters29: 1524–36.
26.
ZhangC-XZhangJ-S (2008b) A local boosting algorithm for solving classification problems. Computational Statistics & Data Analysis52: 1928–41.
27.
ZhangC-XZhangJ-SZhangG-Z (2008) An efficient modified boosting method for solving classification problems. Journal of Computational and Applied Mathematics214: 381–92.
28.
ZhuX-qYangY (2008) A lazy bagging approach to classification. Pattern Recognition41: 2980–92.
