Maximum-entropy estimated distribution model for classification problems

Abstract

Classification is a fundamental problem in machine learning and data mining. This paper applies a stochastic optimization model to classification problems. The proposed maximum entropy estimated distribution model uses a probabilistic distribution to represent solution space, and a sampling technique to explore search space. This paper demonstrates the application of the proposed maximum entropy estimated distribution model to improve linear discriminant function and rule induction methods. In addition, this paper compares the proposed classification model with decision trees. It shows that the proposed model is preferable to decision tree C4.5 in the following cases: i) when prior distribution of classification is available; ii) when no assumption is made about underlying classification structure; and iii) when a classification problem is multimodal in nature.