Improved subspace-based and angle-based outlier detections for fuzzy datasets with a real case study

Abstract

To overcome curse of dimensionality for outlier detecting in high dimensional dataset, axis-parallel subspace (SOD) and angle-based outlier detection (ABOD) methods were presented. These methods are also friendly used distance-based to detect outliers. In this regard, based on the reality of fuzzy data for explaining the world phenomena, this paper introduces an extended version of both methods for fuzzy dataset. First, the basic concepts of both methods are explained. Next we provide two metrics based on Euclidean and analytic distance to measure distance between fuzzy objects; also Cosine similarity measure formula for calculating the cosine of angle between two difference vectors in high-dimensional fuzzy dataset is illustrated. Then the algorithms to determine outliers of fuzzy datasets by using these metrics and Cosine similarity measure, based on ABOD and SOD algorithms, are presented. Some numerical experimental examples are also presented, in which both real and synthesis datasets are used, For a real numerical examination, we have applied proposed algorithms to data from 15 Iranian petrochemical companies in a fully fuzzy environment. The obtained results show the significant properties of the new methods in detecting outliers.

Keywords

Outlier detection angle-based outlier detection axis-parallel subspace fuzzy number cosine-similarity

Get full access to this article

View all access options for this article.

References

Alrezaamiri

, Ebrahimnejad

and Motameni

, Solving the nextrelease problem by means of the fuzzy logic inference system withrespect to the competitive market, Journal of Experimental &Theoretical Artificial Intelligence 32(6) (2020), 959–976.

Angiulli

and Pizzuti

, Fast outlier detection in high dimensional spaces, In European conference on principles of data mining and knowledge discovery, pages 15–27. Springer, (2002).

Bhattacharyya

, On a measure of divergence between two multinomial populations, Sankhyā: the indian journal of statistics, (1946), 401–406.

Breunig

M.M.

, Kriegel

H.-P.

, Ng

R.T.

and Sander

, Lof: identifying density-based local outliers, In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, (2000), 93–104.

Dillon

, Introduction to modern information retrieval: G. salton and m. mcgill. mcgraw-hill, new york (1983). xv+ 448 pp., AA32.95 isbn 0-07-054484-0, (1983).

Fakharzadeh Jahromi

and Ebrahimi Mimand

, A new outlierdetection method for high dimensional fuzzy databases based on lof, Journal of Mathematical Modeling 6(2) (2018), 123–136.

Fakharzadeh Jahromi

and Zarei

, A loop based outlier detectionmethod for high dimensional fuzzy data set, Journal ofIntelligent & Fuzzy Systems 32(1) (2017), 241–248.

Gebremeskel

G.B.

, Yi

, He

and Haile

, Combined data mining techniques based patient data outlier detection for healthcare safety, International Journal of Intelligent Computing and Cybernetics (2016).

Huang

, Kong

and Chen

, Outlier discrimination and correction in intelligent transportation systems, In Smart Cities Cybersecurity and Privacy, pages 203–215. Elsevier, (2019).

10.

Jarvis

R.A.

and Patrick

E.A.

, Clustering using a similarity measurebased on shared near neighbors,–, IEEE Transactions on computers 100(11) (1034).

11.

John

and Naaz

, Credit card fraud detection using local outlierfactor and isolation forest, Int. J. Comput. Sci. Eng 7(4) (2019), 1060–1064.

12.

Kriegel

H.-P.

, Kröger

, Schubert

and Zimek

, Outlier detection in axis-parallel subspaces of high dimensional data. In Pacific-asia conference on knowledge discovery and data mining, pages 831–838. Springer, (2009).

13.

Kriegel

H.-P.

, Schubert

and Zimek

, Angle-based outlier detection in high-dimensional data, In Proceedings of the 14th ACMSIGKDD international conference on Knowledge discovery and data mining, pages (2008), 444–452.

14.

Latpate

R.V.

, Fuzzy modified topsis for supplier selection problemin supply chain management, International Journal of InnovativeResearch in Computer Science & Technology 3(4) (2015), 22–28.

15.

Liu

, Outlier detection based on variance of angle in high dimensional data, In Sixth International Conference on Electronics and Information Engineering, volume 9794, page 979407. International Society for Optics and Photonics, (2015).

16.

Luo

and Cheng

, The distance between fuzzy sets in fuzzy metric spaces. In 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pages 190–194. IEEE, (2015).

17.

Mohseni

and Fakharzade

, A new local distance-based outlier detection approach for fuzzy data by vertex metric. In 2015 2nd International Conference on Knowledge-Based Engineering and Innovation (KBEI), pages 551–554. IEEE, (2015).

18.

Mozaffari

M.R.

, Mohammadi

, Wanke

P.F.

and Correa

H.L.

, Towards greener petrochemical production: Two-stage network data envelopmentanalysis in a fully fuzzy environment in the presence of undesirableoutputs, Expert Systems with Applications 164 (2021).

19.

Papadimitriou

, Kitagawa

, Gibbons

P.B.

and Faloutsos

, Loci: Fast outlier detection using the local correlation integral. In Proceedings 19th international conference on data engineering (Cat. No. 03CH37405), pages 315–326. IEEE, (2003).

20.

Pham

and Pagh

, A near-linear time approximation algorithm for angle-based outlier detection in high-dimensional data. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, (2012), 877–885.

21.

Rehman

M.U.

and Khan

D.M.

, Local neighborhood-based outlier detection of high dimensional data using different proximity functions, In International Journal of Advanced Computer Science and Applications (IJACSA), (2020), 133–137.

22.

Sandosh

, Govindasamy

and Akila

, Enhanced intrusiondetection system via agent clustering and classification based onoutlier detection, Peer-to-Peer Networking and Applications 13(3) (2020), 1038–1045.

23.

Zhang

T.G.

and Zhao

X.L.

, Web service selection based on fuzzy synthetic evaluation, In Advanced Materials Research, volume 187, pages 227–231. Trans Tech Publ, (2011).