A novel three layer particle swarm optimization for feature selection

Abstract

Feature selection (FS) is a vital data preprocessing task which aims at selecting a small subset of features while maintaining a high level of classification accuracy. FS is a challenging optimization problem due to the large search space and the existence of local optimal solutions. Particle swarm optimization (PSO) is a promising technique in selecting optimal feature subset due to its rapid convergence speed and global search ability. But PSO suffers from stagnation or premature convergence in complex FS problems. In this paper, a novel three layer PSO (TLPSO) is proposed for solving FS problem. In the TLPSO, the particles in the swarm are divided into three layers according to their evolution status and particles in different layers are treated differently to fully investigate their potential. Instead of learning from those historical best positions, the TLPSO uses a random learning exemplar selection strategy to enrich the searching behavior of the swarm and enhance the population diversity. Further, a local search operator based on the Gaussian distribution is performed on the elite particles to improve the exploitation ability. Therefore, TLPSO is able to keep a balance between population diversity and convergence speed. Extensive comparisons with seven state-of-the-art meta-heuristic based FS methods are conducted on 18 datasets. The experimental results demonstrate the competitive and reliable performance of TLPSO in terms of improving the classification accuracy and reducing the number of features.