Sage Journals: Discover world-class research

Abstract

The Support Vector Machine (SVM) is a powerful technique for data classification. For linearly separable data points, the SVM constructs an optimal separating hyper-plane as a decision surface, to divide the data points of different categories in the vector space. For the non-linearly separable data points, the Kernel functions are used to extend the concept of the optimal separating hyper-plane so that the data can be linearly separable. The different kernel functions have different characteristics and hence the performance of the SVM is highly influenced by the selection of kernel functions. This paper presents the classification algorithm that uses the SVM in the training phase and the Mahalanobis distance in the testing phase, in order to design a classifier which has low impact of kernel function on the classification accuracy, positively. The Mahalanobis distance is used to replace the optimal separating hyper-plane as the classification decision making function in the SVM. The proposed approach is compared with Euclidean-SVM, which uses Euclidean distance function to replace the optimal separating hyper-plane as the classification boundary. It has also been evaluated against conventional SVM too. The experimental results show that the accuracy of the EuDiC (Euclidean Distance towards the Center of data) SVM classifier has a low impact on the implementation of kernel functions. The EuDiC SVM also achieves the drastic reduction in the classification time since it only depends on the mean of Support Vectors(SVs) of each category for classification. To prove its effectiveness on other types of data, the time series data have also been used. Due to robust design of the EuDiC, it also performs well for time series data too.

Keywords

Classification Euclidean distance kernel function Mahalanobis distance optimal hyper-plane support vector machine support vector

Get full access to this article

View all access options for this article.

References

Nanopoulos

, Alcock

and Manolopoulos

, Feature-based classification of time-series data, International Journal of Computer Research 10(3) (2001).

Quang

A.T.

, Zhang

Q.L.

and Li

, Evolving support vector machine parameters, in: Proceedings of International Conference on Machine Learning and Cybernetics, IEEE 1 (2002), 548-551.

Boser

B.E.

, Guyon

I.M.

and Vapnik

V.N.

, A training algorithm for optimal margin classifiers, in: Proceedings of the Fifth Annual Workshop on Computational Learning Theory ACM (Jul 1992), 144-152.

Haasdonk

and Pęekalska

, Classification with kernel Mahalanobis distance classifiers, in: Advances in Data Analysis, Data Handling and Business Intelligence, Springer Berlin Heidelberg (2010), 351-361.

Schölkopf

and Smola

A.J.

, Learning with kernels: Support vector machines, regularization, optimization and beyond, MIT Press, 2002.

Aggarwal

C.C.

, Hinneburg

and Keim

D.A.

, On the surprising behavior of distance metrics in high dimensional space, Springer Berlin Heidelberg (2001), 420-434.

Chang

C.C.

and Lin

C.J.

, LIBSVM: A library for support vector machines, ACM Transactions on Intelligent Systems and Technology (TIST) 2(3) (2011), 27.

Campbell

and Ying

, Learning with support vector machines, Synthesis Lectures on Artificial Intelligence and Machine Learning 5(1) (2011), 1-95.

Cortes

and Vapnik

, Support-vector networks, Machine Learning 20(3) (1995), 273-297.

10.

Burges

C.J.

, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery 2(2) (1998), 121-167.

11.

Staelin

, Parameter selection for support vector machines, Hewlett-Packard Company, Tech Rep HPL-2002-354R1, 2003.

12.

Eads

D.R.

, Hill

, Davis

, Perkins

S.J.

, Ma

, Porter

R.B.

and Theiler

J.P.

, Genetic algorithms and support vector machines for time series classification, in: International Symposium on Optical Science and Technology, International Society for Optics and Photonics (Dec 2002), 74-85.

13.

Wang

, Yeung

D.S.

and Tsang

E.C.

, Weighted Mahalanobis distance kernels for support vector machines, IEEE Transactions on Neural Networks 18(5) (2007), 1453-1462.

14.

Mclachlan

G.J.

, Mahalanobis distance, Resonance 46 (1999), 20-26.

15.

Bhavsar

H.B.

and Jivani

A.G.

, The shared nearest neighbor algorithm with enclosures (SNNAE), in: Computer Science and Information Engineering, 2009 WRI World Congress on, IEEE 4 (Mar 2009), 436-442.

16.

Bhavsar

and Ganatra

, A comparative study of training algorithms for supervised machine learning, International Journal of Soft Computing and Engineering (IJSCE) 2(4) (2012), 2231-2307.

17.

Bhavsar

and Ganatra

, Increasing efficiency of support vector machine using the novel kernel function: combination of polynomial and radial basis function, International Journal on Advanced Computer Theory and Engineering (IJACTE) 3(5) (2014), 17-54.

18.

Song

, Ding

, Guo

, Li

and Xia

, Research on combination kernel function of support vector machine, in: International Conference on Computer Science and Software Engineering, 2008, IEEE 1 (Dec 2008), 838-841.

19.

Han

, Kamber

and Pei

, Data mining: Concepts and techniques: Concepts and techniques, Elsevier, 2011.

20.

Moguerza

J.M.

and Muñoz

, Support vector machines with applications, Statistical Science 21(3) (2006), 322-336.

21.

Platt

, Fast training of support vector machines using sequential minimal optimization, Advances in Kernel Methods - Support Vector Learning 3 (1999).

22.

Sun

, Fast tuning of SVM kernel parameter using distance between two classes, in: 3rd International Conference on Intelligent System and Knowledge Engineering, 2008 ISKE, IEEE 1 (Nov 2008), 108-113.

23.

Chan

K.P.

and Fu

A.W.C.

, Efficient time series matching by wavelete, in: Proceedings of 15th IEEE International Conference on Data Engineering (Mar 1999), 126-133.

24.

K.P.

and Wang

S.D.

, Choosing the kernel parameters for support vector machines by the inter-cluster distance in the feature space, Pattern Recognition 42(5) (2009), 710-717.

25.

Weinberger

K.Q.

, Blitzer

and Saul

L.K.

, Distance metric learning for large margin nearest neighbor classification, in: The Journal of Machine Learning Research (2009), 207-244.

26.

Müller

K.R.

, Mika

, Rätsch

, Tsuda

and Schölkopf

, An introduction to kernel-based learning algorithms, IEEE Transactions on Neural Networks 12(2) (2001), 181-201.

27.

Lee

L.H.

, Wan

C.H.

, Rajkumar

and Isa

, An enhanced support vector machine classification framework by using Euclidean distance function for text document categorization, Applied Intelligence 37(1) (2012), 80-99.

28.

Yang

and Jin

, Distance metric learning: A comprehensive survey, Michigan State Universiy 2 (2006).

29.

Fauvel

, Villa

, Chanussot

and Benediktsson

J.A.

, Mahalanobis kernel for the classification of hyperspectral images, IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (Jul 2010), 3724-3727.

30.

Sundaram

, Support vector machine approximation using kernel PCA, Univ California At Berkeley, Berkeley, CA, USA, Tech Rep UCB/EECS-2009-94, 2009. Available: http://wwweecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-94.pdf.

31.

De Maesschalck

, Jouan-Rimbaud

and Massart

D.L.

, The Mahalanobis distance, Chemometrics and Intelligent Laboratory System 50(1) (2000), 1-18.

32.

Herbrich

, Learning kernel classifiers, MIT Press, Cambridge, 2002, pp. 17-110.

33.

Kotsiantis

S.B.

, Supervised machine learning: A review of classification techniques, Informatica 31 (2007), 249-268.

34.

Cha

, Comprehensive survey on distance/similarity measures between probability density functions, International Journal of Mathamatical Models and Method of Applied Sciencce 4(1) (2007), 300-307.

35.

Xiang

, Nie

and Zhang

, Learning a Mahalanobis distance metric for data clustering and classification, Pattern Recognition 41(12) (2008), 3600-3612.

36.

Phyu

T.N.

, Survey of classification techniques in data mining, in: Proceedings of the International MultiConference of Engineers and Computer Scientists 1 (Mar 2009), 18-20.

37.

Wang

, Improving SVM classification by feature weight learning, in: International Conference on Intelligent Computation Technology and Automation (ICICTA), 2010, IEEE 2 (May 2010), 518-521.

38.

Peng

and Xu

, Twin Mahalanobis distance-based support vector machines for pattern recognition Information Sciences 200 (2012), 22-37.

39.

Wei

, Liu

, Zhan

and Li

, Mahalanobis support vector machines made fast and robust, INTECH Open Access Publisher, 2010.

40.

Y.L.

, Agrawal

and El Abbadi

, A comparison of DFT and DWT based similarity search in time-series databases, in: Proceedings of the Ninth International Conference on Information and Knowledge Management, ACM, (Nov 2000), 488-495.

EuDiC SVM: A novel support vector machine classification algorithm

Abstract

Keywords

Get full access to this article

References