Random subspace oracle (RSO) ensemble to solve small sample-sized classification problems

Abstract

Under certain situations, researchers were forced to work with small sample-sized (SSS) data. With very limited sample size, SSS data have the tendency to undertrain a machine learning algorithm and rendered it ineffective. Some extreme cases in SSS problems will have to deal with large feature-to-instance ratio, where the high number of features compared to small number of instances will overfit the classification algorithm. This paper intends to solve small sample-sized classification problems through hybrid of random subspace method and random linear oracle ensemble by utilizing binary feature subspace splitting and oracle selection scheme. Experimental results on artificial data indicate the proposed algorithm can outperform single decision tree and linear discriminant classifiers in small sample-sized data, but its performance is identical to k-nearest neighbor classifier due to both shared similar selection approach. Results from real-world medical data indicate the proposed method has better classification performance than its corresponding single base classifier especially in the case of decision tree.