Ensemble based on feature projection and under-sampling for imbalanced learning

Abstract

Imbalanced problem is concerned with the performance of classifiers on the data set with severe class imbalance distribution. For two-class, the examples can be categorized into majority class or minority class, and the cost of misclassifying minority class examples is often much higher than the contrary cases. However, traditional classifiers do not work well on the imbalanced problem due to the assumption that the number of each class examples is similar to each other. To handle these problems, this paper proposes a simple but effective ensemble learning method based on feature projection and under-sampling (EFPUS). EFPUS learns an ensemble through the following two steps: (1) under-sampling several subset from majority class and learning a novel projection matrix from each subset, and (2) constructing new training sets by projecting the original training set to different spaces defined by the matrixes and learning a class-imbalance oriented classifier from each new training set. For the first step, feature projection and under-sampling mainly aim to improve the diversity between ensemble members. With respect to the second step, the base models can be learned by any traditional class-imbalance oriented learning method such as USBagging, SMOTEBoost and BalanceCascade. Experimental results show that, compared with other state-of-the-art methods, EFPUS shows significantly better performance on measures of g-mean, f-measure, AUC, recall and accuracy.