Feature selection is a crucial data pre-processing step in classification problems. The wrapper approach is widely used due to their good classification performance. However, it is very computational expensive due to the cross validation scheme in the evaluation phase. In order to solve this problem, this paper proposes a novel hybrid two-stage feature selection method based on differential evolution (HTSDE). In the first stage, a cluster validity index named DB index is employed to evaluate the feature subset and the wrapper approach in used in the second stage to improve the classification accuracy of the feature subsets. In order to find global optimal feature subsets, different trail vector generation strategies of DE are used in the two stages where the first stage focuses on global exploration and the second stage emphasizes fast convergence. The hybrid method is able to combine the advantages of both DB index and wrapper approach and improve the computational efficiency of the wrapper approach while maintaining the classification performance. HTSDE is compared with several state-of-the-art feature selection methods on 12 datasets. Experimental results show the proposed HTSDE achieves higher classification accuracy than both wrapper and filter approaches. Moreover, its computational cost is much less than those wrapper approaches.
LiY., LiT. and LiuH., Recent advances in feature selection and its applications, Knowledge and Information Systems53(3) (2017), 551–577.
2.
DashM. and LiuH., Feature selection for classification, Intelligent Data Analysis1(1–4) (1997), 131–156.
3.
GuyonI. and ElisseeffA., An introduction to variable and feature selection, Journal of Machine Learning Research3 (2003), 1157–1182.
4.
LiuH. and YuL., Toward integrating feature selection algorithms for classification and clustering, IEEE Trans Knowl Data Eng17(4) (2005), 491–502.
5.
GuyonI.M., GunnS.R., NikraveshM., et al., Feature Extraction: Foundations and Applications, 207 Springer, (2008).
6.
NguyenH., XueB., LiuI., et al., Filter based backward elimination in wrapper based PSO for feature selection in classification, in Proceedings of the IEEE Congress on Evolutionary Computation (2014), 3111–3118.
7.
XueB., ZhangM.J., BrowneW.N. and YaoX., A survey on evolutionary computation approaches to feature selection, IEEE Trans Evol Comput20(4) (2016), 606–626.
8.
KabirM., ShahjahanM. and MuraseK., A new local search based hybrid genetic algorithm for feature selection, Neurocomputing74(17) (2011), 2914–2928.
9.
HuangJ., CaiY. and XuX., A hybrid genetic algorithm for feature selection wrapper based on mutual information, Patt Recog Lett Arch28(13) (2007).
10.
XueB., ZhangM. and BrowneW.N., Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms, Appl Soft Comput18 (2014), 261–276.
11.
MoradiP. and GholampourM., A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy, Applied Soft Computing43 (2016), 117–130.
12.
QiuC., Bare bones particle swarm optimization with adaptive chaotic jump for feature selection in classification, International Journal of Computational Intelligence Systems11(1) (2018), 1–14.
13.
QiuC. and ZuoX., Barebones Particle Swarm Optimization with a Neighborhood Search Strategy for Feature Selection, in Proceedings of International Conference on Bio-Inspired Computing: Theories and Applications, (2018), pp, 42–54.
14.
KashefS. and NezamabadiH., An advanced ACO algorithm for feature subset selection, Neurocomputing147 (2015), 271–279.
15.
WanY., WangM., YeZ. and LaiX., A feature selection method based on modified binary coded ant colony optimization algorithm, Applied Soft Computing49 (2016), 248–258.
16.
BhadraT. and BandyopadhyayS., Unsupervised feature selection using an improved version of Differential Evolution, , Expert Systems with Applications42 (2015), 4042–4053.
17.
LiT., DongH. and SunJ., Binary Differential Evolution Based on Individual Entropy for Feature Subset Optimization, IEEE Access7 (2019), 24109–24121.
18.
HancerE., XueB. and ZhangM., Differential evolution for filter feature selection based on information theory and feature ranking, Knowledge-Based Systems140 (2018), 103–119.
19.
HancerE., XueB., KarabogaD., et al., A binary ABC algorithm based on advanced similarity scheme for feature selection, Applied Soft Computing36 (2015), 334–348.
20.
GhaemiM. and DerakhshiM., Feature selection using Forest Optimization Algorithm, Pattern Recognition60 (2016), 121–129.
21.
FarisH., MafarjaM., HeidariA., et al., An efficient binary Salp Swarm Algorithm with crossover scheme for feature selection problems, Knowledge-Based Systems154 (2018), 43–67.
22.
EmaryE., ZawbaaH.M. and HassanienA.E., Binary grey wolf optimization approaches for feature selection, Neurocomputing172 (2016), 371–381.
23.
MarillT. and GreenD., On the effectiveness of receptors in recognition systems, IEEE Trans Inf Theory9(1) (2006), 11–17.
24.
MoradiP. and GholampourM., A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy, Applied Soft Computing43 (2016), 117–130.
25.
ZorarpaciE. and ÖzelS.A., A hybrid approach of differential evolution and artificial bee colony for feature selection, Expert Systems With Applications62 (2016), 91–103.
26.
NguyenH.B., XueB., LiuI., et al., Filter based backward elimination in wrapper based pso for feature selection in classification, in: IEEE Congress on Evolutionary Computation (CEC’2014), (2014), pp. 3111–3118.
27.
QiuC. and XiangF., Feature selection using a set based discrete particle swarm optimization and a novel feature subset evaluation criterion, Intelli Data Ana23(1) (2019), 5–21.
28.
XueB., CervanteL., ShangL., et al., Binary pso and rough set theory for feature selection: a multi-objective filter based approach, Comput Intell Appl13(2) (2014), 1450009.
29.
StornR. and PriceK.V., Differential evolution: A simple and efficient heuristic for global optimization over continuous spaces, J Global Opt11(4) (1997), 341–359.
30.
ArbelaitzO., GurrutxagaI., MuguerzaJ., et al., An extensive comparative study of cluster validity indices, Pattern Recognition46 (2013), 243–256.
31.
LiuZ., WangY. and YangS., Differential evolution with a two-stage optimization mechanism for numerical optimization, in: Proceedings of IEEE Congress on Evolutionary Computation, (2016), pp. 3170–3177.
32.
QinA.K., HuangV.L. and SuganthanP.N., Differential evolution algorithm with strategy adaptation for global numerical optimization, IEEE Trans Evol Comput13(2) (2009), 398–417.
33.
DaviesD.L. and BouldinD.W., A clustering separation measure, IEEE Transactions on Pattern Analysis and Machine Intelligence1 (1979), 224–227.
34.
CaiH., ChungC. and WongK., Application of differential evolution algorithm for transient stability constrained optimal power flow, IEEE Trans. Power Syst23(484) (2008), 719–728.
35.
DashR., DashP. and BisoiR., A self adaptive differential harmony search based optimized extreme learning machine for financial time series prediction, Swarm Evol Comput (2014), 25–42.
36.
GhoshP., DasS. and ZafarH., Adaptive differential evolution based design of two-channel quadrature mirror filter banks for sub-band coding and data transmission, IEEE Trans Syst Man Cybern C Appl Rev42(6) (2012), 1613–1623.
37.
LiuW. and WangJ., A Brief Survey on Nature-Inspired Metaheuristics for Feature Selection in Classification in this Decade, in Proceedings of 2019 IEEE 16th International Conference on Networking, Sensing and Control (ICNSC), (2019), pp, 424–429.
38.
ChakrabortyB., Genetic algorithm with fuzzy fitness function for feature selection, in Proc IEEE ISIE, (2002), pp. 315–319.
39.
DasA. and DasD., Feature weighting and selection with a Pareto-optimal trade-off between relevancy and redundancy, Pattern Recognition Letters88 (2017), 12–19.
40.
QiuC., A novel multi-swarm particle swarm optimization for feature selection, Genetic Programming and Evolvable Machines20(4) (2019), 503–529.
41.
NguyenH., XueB., AndreaeP. and ZhangM., Particle Swarm Optimisation with genetic operators for feature selection. In Proceedings of IEEE International Conference on Evolutionary Computations (2017), pp. 286–293.
42.
TabakhiS. and MoradiP., Relevance–redundancy feature selection based on ant colony optimization, Pattern Recognition48 (2015), 2798–2811.
43.
AladeemyM., AdwanL., BoothA., et al., New feature selection methods based on opposition-based learning and self-adaptive cohort intelligence for predicting patient no-shows, Applied Soft Computing86 (2020), 105866.
44.
TubishatM., IdrisN. and ShuibL., Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection, Expert Systems with Applications145(1) (2020), 113–122.
45.
SayedG., KhoribaG. and HaggagM., A novel chaotic salp swarm algorithm for global optimization and feature selection, Applied Intelligence48(10) (2018), 3462–3481.
46.
MafarjaM., AljarahI., FarisH., et al., Binary grasshopper optimisation algorithm approaches for feature selection problems, Expert Systems With Applications117 (2019), 267–286.
47.
LiuC., WangW., ZhaoQ., et al., A new feature selection method based on a validity index of feature subset, Pattern Recognition Letters92 (2017), 1–8.
48.
WangY., CaiZ. and ZhangQ.F., Differential evolution with composite trial vector generation strategies and control parameters, IEEE Trans Evol Comput15(1) (2011), 55–67.
49.
MallipeddiR., SuganthanP.N., PanQ., et al., Differential evolution algorithm with ensemble of parameters and mutation strategies, Appl Soft Comput11(2) (2011), 1679–1696.
TranB., ZhangM. and XueB., A PSO based hybrid feature selection algorithm for high-dimensional classification. in Proc. 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 3801–3808.
52.
ZhangY., GongD., HuY., et al., Feature selection algorithm based on bare bones particle swarm optimization, Neurocomputing148 (2015), 150–157.
53.
GuS., ChengR. and JinY., Feature selection for high-dimensional classification using a competitive swarm optimizer, Soft Computing22(3) (2018), 811–822.
54.
PengH., LongF. and DingC., Feature selection based on mutual information criteria of max-dependency, max-relevance, and min redundancy, IEEE Trans. Pattern Anal. Mach. Intell27(8) (2005), 1226–1238.
55.
BattitiR., Using mutual information for selecting features in supervised neural net learning, IEEE Trans Neural Netw5(4) (1994), 537–550.
56.
YangH. and MoodyJ., Data visualization and feature selection: New algorithms for non-gaussian data, Advances Neural Inf Process Syst12 (1999), 687–693.
