Feature selection is the problem of eliminating the features which are irrelevant and/or redundant. It can also be assumed as the problem of selecting a small subset of features which are necessary and sufficient to describe the target concept. In this paper, a new feature selection method based on the concepts of sensitivity and Pearson’s correlation is introduced which is called Sensitivity and Correlation based Feature Selection-SCFS. The sensitivity of one feature is computed via applying the subtractive clustering and is utilized as feature-target relevancy. Pearson’s correlation coefficient is used to determine the redundancy among a subset of selected features. The introduced measure increases the score of a selected feature subset which has maximum relevancy to the target concept and minimum redundancy among features. The proposed criterion is employed as the fitness function in a genetic algorithm in order to evaluate feature subsets. Some well-known benchmark datasets are utilized for investigating the performance of the proposed method. Also, the results of our method are compared with the other similar feature selection methods. The obtained results show however SCFS is an unsupervised filter; it is well comparable to the other well-known supervised methods in terms of classification accuracy and the number of selected features.
YanH., YuanX., YanS. and YangJ., Correntropy based feature selection using binary projection, Pattern Recognition44 (2011), 2834–2842.
2.
ZhengH. and ZhangY., Feature selection for high-dimensional data in astronomy, Advances in Space Research41 (2008), 1960–1964.
3.
IlbeygiM. and Shah-HosseiniH., A novel fuzzy facial expression recognition system based on facial feature extraction from color face images, Engineering Applications of Artificial Intelligence25 (2012), 130–146.
4.
VicientC., SánchezD. and
MorenoA., An automatic approach for ontology-based feature extraction from heterogeneous textualresources, Engineering Applications of Artificial Intelligence26 (2013), 1092–1106.
5.
MohammadiF.G. and AbadehM.S., Image steganalysis using a bee colony based feature selection algorithm, Engineering Applications of Artificial Intelligence31 (2014), 35–43.
6.
TabakhiS., MoradiP. and AkhlaghianF., An unsupervised feature selection algorithm based on ant colony optimization, Engineering Applications of Artificial Intelligence32 (2014), 112–123.
7.
KabirM. and IslamM., A new wrapper feature selection approach using neural network, Neurocomputing73 (2010), 3273–3283.
8.
VieiraS.M., SousaJ. and RunklerT.A., Two cooperative ant colonies for feature selection using fuzzy models, Expert Systems with Applications37 (2010), 2714–2723.
9.
LuukkaP., Feature selection using fuzzy entropy measures with similarity classifier, Expert Systems with Applications38 (2011), 4600–4607.
10.
EsfandianN., RazzaziF. and BehradA., A clustering based feature selection method in spectro-temporal domain for speech recognition, Engineering Applications of Artificial Intelligence25 (2012), 1194–1202.
11.
LeeC. and LeeG.G., Information gain and divergence-based feature selection for machine learning-based text categorization, Information Processing & Management42 (2006), 155–165.
12.
ZibakhshA. and AbadehM.S., Gene selection for cancer tumor detection using a novel memetic algorithm with a multi-view fitness function, Engineering Applications of Artificial Intelligence26 (2013), 1274–1281.
13.
ChakrabortiT. and ChatterjeeA., A novel binary adaptive weight GSA based feature selection for face recognition using local gradient patterns, modified census transform, and local binary patterns, Engineering Applications of Artificial Intelligence33 (2014), 80–90.
14.
Ajit KrisshnaN.,
DeepakV.K.,
ManikantanK. and RamachandranS., Face recognition using transform domain feature extraction and PSO-based feature selection, Applied Soft Computing (2014).
LinH.-Y., Feature selection based on cluster and variability analyses for ordinal multi-class classification problems, Knowledge-Based Systems (2012).
17.
SunX., LiuY., LiJ., ZhuJ., LiuX. and ChenH., Using cooperative game theory to optimize the feature selection problem, Neurocomputing (2012).
18.
UnlerA., MuratA. and ChinnamR.B., mr2 PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification, Information Sciences181 (2011), 4625–4641.
19.
SikoraR. and PiramuthuS., Framework for efficient feature selection in genetic algorithm based data mining, European Journal of Operational Research180 (2007), 723–737.
20.
CaiR., HaoZ., YangX. and WenW., An efficient gene selection algorithm based on mutual information, Neurocomputing72 (2009), 991–999.
21.
ChenD., HuQ. and YangY., Parameterized attribute reduction with Gaussian kernel based fuzzy rough sets, Information Sciences181 (2011), 5169–5179.
22.
CovoesT.F. and HruschkaE.R., Towards improving cluster-based feature selection with a simplified silhouette filter, Information Sciences181 (2011), 3766–3782.
23.
HallM.A., Correlation-based feature selection for machine learning, The University of Waikato, 1999.
24.
DashM. and LiuH., Consistency-based search in feature selection, Artificial Intelligence151 (2003), 155–176.
25.
LiuR. and ShiY., Spatial distance join based feature selection, Engineering Applications of Artificial Intelligence26 (2013), 2597–2607.
26.
PengH., LongF. and DingC., Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, Pattern Analysis and Machine Intelligence, IEEE Transactions on27 (2005), 1226–1238.
27.
HamdaniT.M., WonJ.-M., AlimiA.M. and KarrayF., Hierarchical genetic algorithm with new evaluation function and bi-coded representation for the selection of features considering their confidence rate, Applied Soft Computing11 (2011), 2501–2509.
28.
WeiL., Asymptotic conservativeness and efficiency of Kruskal-Wallis test for K dependent samples, Journal of the American Statistical Association76 (1981), 1006–1009.
29.
ShangW., HuangH., ZhuH., LinY., QuY. and WangZ., A novel feature selection algorithm for text categorization, Expert Systems with Applications33 (2007), 1–5.
30.
RaileanuL.E. and StoffelK., Theoretical comparison between the gini index and information gain criteria, Annals of Mathematics and Artificial Intelligence41 (2004), 77–93.
31.
CawleyG.C. and TalbotN.L., Gene selection in cancer classification using sparse logistic regression with Bayesian regularization, Bioinformatics22 (2006), 2348–2355.
32.
GuQ., LiZ. and HanJ., Generalized fisher score for feature selection, arXiv preprint arXiv:1202.3725, 2012.
33.
KononenkoI., Estimating attributes: Analysis and extensions of RELIEF, Machine Learning: ECML-94, Springer, 1994, pp. 171–182.
34.
LiuH. and SetionoR., Chi2: Feature selection and discretization of numeric attributes, Tools with Artificial Intelligence, 1995 Proceedings, Seventh International Conference on, IEEE, 1995, pp. 388–391.
35.
CawleyG.C., TalbotN.L. and GirolamiM., Sparse multinomial logistic regression via bayesian l1 regularisation, Advances in Neural Information Processing Systems19 (2007), 209.
36.
YuL. and LiuH., Feature selection for high-dimensional data: A fast correlation-based filter solution, Machine Learning-International Workshop Then Conference (2003), p. 856.
37.
SedgwickP., Pearson’s correlation coefficient, BMJ: British Medical Journal345 (2012).
38.
TsaiC.F., EberleW. and ChuC.-Y., Genetic algorithms in feature and instance selection, Knowledge-Based Systems (2012).
39.
KudoM. and SklanskyJ., Comparison of algorithms that select features for pattern classifiers, Pattern Recognition33 (2000), 25–41.
40.
JangJ.S.R. and SunC.T., Neuro-fuzzy and soft computing: A computational approach to learning and machine intelligence, Inc, 1996–Prentice-Hall.
41.
IliadisL., VangeloudhM. and SpartalisS., An intelligent system employing an enhanced fuzzy c-means clustering model: Application in the case of forest fires, Computers and Electronics in Agriculture70 (2010), 276–284.
42.
ChiuS.L., Fuzzy model identification based on cluster estimation, Journal of intelligent and Fuzzy systems2 (1994), 267–278.
43.
HammoudaK. and KarrayF., A comparative study of data clustering techniques, Tools of Intelligent Systems Design Course Project SYDE625 (2000).
44.
JangJ.-S.R., SunC.-T. and MizutaniE., Neuro-fuzzy and soft computing-a computational approach to learning and machine intelligence [Book Review], Automatic Control IEEE Transactions on42 (1997), 1482–1484.
45.
AsuncionA. and NewmanD., UCI machine learning repository, 2007.
46.
OugiaroglouS., NanopoulosA., PapadopoulosA.N., ManolopoulosY. and Welzer-DruzovecT., Adaptive k-nearest-neighbor classification using a dynamic number of nearest neighbors, Advances in Databases and Information Systems, Springer, 2007, pp. 66–82.
47.
GarciaS. and HerreraF., An extension on statistical comparisons of classifiers over multiple data sets for all pairwise comons, Journal of Machine Learning Research9 (2008), 2677–2694.
48.
AlcaláJ.,
FernándezA.,
LuengoJ.,
DerracJ.,
GarcíaS.,
SánchezL. and
HerreraF., KEEL data-mining software tool, Data set repository, integration of algorithms and experimental analysis framework, Journal of Multiple-Valued Logic and Soft Computing (2010).
49.
NskiB.T., EtekM.S., TelecZ. and LasotaT., Nonparametric statistical analysis for multiple comparison of machine learning regression algorithms, Int J Appl Math Comput Sci22 (2012), 867–881.
