An evolving Takagi-Sugeno model based on aggregated trapezium clouds for anomaly detection in large datasets

Abstract

Anomaly detection is an important task for applications involving Big Data. Comparing with traditional method, anomaly detection in Big Data confronts growing amounts of data with high dimensionality and complex structures, which require more real-time analysis. This paper presents a fuzzy input-output system for anomalous data using electronic consumer records (ECR), a trapezium-cloud-map-filtration (TCMF) framework and a value mining model. ECRs are used to add or remove criteria based on consumers’ consumption. In addition, MapReduce framework and trapezium clouds generated from each subsample are aggregated by using the aggregated trapezium cloud as a filter for each subsample. Then, a fuzzy logic-based value mining model is proposed based on Takagi-Sugeno model (T-S model) and trapezium clouds. This paper establishes a system that can improve decision-making accuracy by filtering large-scale data, and an illustrative example using a hotel booking situation is presented to verify the validity and feasibility of the proposed model. Finally, a comparative analysis is conducted between the proposed approach and existing methods.

Keywords

T-S model Big Data trapezoidal cloud cloud droplet consumers’ classification

Get full access to this article

View all access options for this article.

References

Fan

J.Q.

, Han

and Liu

, Challenges of big data analysis, National Science Review 1(2) (2014), 293–314.

Rajaraman

, Leskovec

and Ullman

, Mining of massive datasets, Cambridge University Press, 2011.

Städler

, Bühlmann

and Geer

S.V.D.

, ℓ₁-penalization for mixture regression models, Test 19(2) (2010), 209–256.

Khalili

and Chen

, Variable selection in finite mixture of regression models, Journal of the American Statistical Association 102(479) (2007), 1025–1038.

X.D.

, Zhu

X.Q.

, Wu

G.Q.

and Ding

, Data mining with big data, IEEE Transactions on Knowledge and Data Engineering 26(1) (2014), 97–107.

Reichman

O.J.

, Jones

M.B.

and Schildhauer

M.P.

, Challenges and opportunities of open data in ecology, Science 331(6018) (2011), 703–705.

Moshtaghi

, Bezdek

J.C.

, Leckie

, Karunasekera

and Palaniswami

, Evolving fuzzy rules for anomaly detection in data streams, IEEE Transactions on Fuzzy Systems 23(3) (2015), 688–700.

Fan

J.Q.

and Lv

J.C.

, Sure independence screening for ultrahigh dimensional feature space, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70(5) (2008), 849–911.

Pyle

, Data Preparation for data mining, Morgan Kaufmann 1 (1999).

10.

Chandola

, Banerjee

and Kumar

, Anomaly detection: A survey, ACM Computing Surveys (CSUR) 41(3) (2009), 15.

11.

Hodge

V.J.

and Austin

, A survey of outlier detection methodologies, Artificial Intelligence Review 22(2) (2004), 85–126.

12.

Knorr

E.M.

, Ng

R.T.

and Tucakov

, Distance-based outliers: Algorithms and applications, The VLDB Journal-The International Journal on Very Large Data Bases 8(3-4) (2000), 237–253.

13.

Kriegel

H.P.

, Kröger

, Schubert

and Zimek

, Outlier detection in axis-parallel subspaces of high dimensional data, In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2009, pp. 831–838.

14.

Zimek

, Schubert

and Kriegel

H.P.

, A survey on unsupervised outlier detection in high-dimensional numerical data, Statistical Analysis and Data Mining: The ASA Data Science Journal 5(5) (2012), 363–387.

15.

, Xu

and Deng

, Discovering cluster-based local outliers, Pattern Recognition Letters 24(9) (2003), 1641–1650.

16.

Campello

R.J.G.B.

, Moulavi

, Zimek

and Sander

, Hierarchical density estimates for data clustering, visualization, and outlier detection, ACM Transactions on Knowledge Discovery from Data (TKDD) 10(1) (2015), 1–51.

17.

Cateni

, Colla

and Vannucci

, A fuzzy logic-based method for outliers detection, In: Artificial Intelligence and Application, 2007, pp. 605–610.

18.

Linda

, Manic

, Vollmer

and Wright

, Fuzzy logic based anomaly detection for embedded network security cyber sensor, In: Computational Intelligence in Cyber Security (CICS), 2011 IEEE Symposium on IEEE, 2011, pp. 202–209.

19.

Last

and Kandel

, Automated detection of outliers in real-world data, In: Proc of the Second International Conference on Intelligent Technologies, 2001, pp. 292–301.

20.

Al-Zoubi

M.D.B.

, Ali

A.-D.

and Yahya

A.A.

, Fuzzy clustering-based approach for outlier detection, In: Proc of the 9th WSEAS Int Conf on Applications of Computer Engineering, 2008, pp. 192–197.

21.

Ishibuchi

, Nakashima

and Nii

, Classification and modeling with linguistic information granules: Advanced approaches to linguistic Data Mining, Springer Science & Business Media, 2006.

22.

Takagi

and Sugeno

, Fuzzy identification of systems and its applications to modeling and control, IEEE Transactions on Systems, Man, and Cybernetics 15(1) (1985), 116–132.

23.

K.L.

and Zakaria

, Electricity consumer classification using artificial intelligence, Universities Power Engineering Conference, 2004, UPEC 2004 39th International 1, 2004, pp. 443–447.

24.

Borguet

and Léonard

, Comparison of adaptive filters for gas turbine performance monitoring, Journal of Computational and Applied Mathematics 234(7) (2010), 2202–2212.

25.

Osborne

M.A.

, Garnett

and Roberts

S.J.

, Active data selection for sensor networks with faults and change points, 2010 24th IEEE International Conference on Advanced Information Networking and Applications, 2010, pp. 533–540.

26.

Ross

G.J.

, Tasoulis

D.K.

and Adams

N.M.

, Online annotation and prediction for regime switching data streams, Proceedings of the 2009 ACM symposium on Applied Computing, 2009, pp. 1501–1505.

27.

Dean

and Ghemawat

, MapReduce: Simplified data processing on large clusters, Communications of the ACM 51(1) (2008), 107–113.

28.

Triguero

, Peralta

, Bacardit

, García

and Herrera

, MRPR: A mapreduce solution for prototype reduction in big data classification, Neurocomputing 150 (2015), 331–345.

29.

López

, Río

S.D.

, Benítez

J.M.

and Herrera

, Cost-sensitive linguistic fuzzy rule based classification systems under the mapreduce framework for imbalanced big data, Fuzzy Sets and Systems 258 (2015), 5–38.

30.

, Fan

, Ma

and Tang

, Energy utilization evaluation of carbon performance in public projects by FAHP and cloud model, Sustainability 8(7) (2016), 630–647.

31.

Zhang

H.Y.

, Zhou

, Wang

and Chen

H.X.

, An FMCDM approach to purchasing decision-making based on cloud model and prospect theory in e-commerce, International Journal of Computational Intelligence Systems 9(4) (2016), 676–688.

32.

Wang

J.Q.

and Yang

W.E.

, Multiple criteria group decision making method based on intuitionistic normal cloud by Monte Carlo simulation, System Engineering Theory and Practice 33(11) (2013), 2859–2865.

33.

Tian

Z.P.

, Wang

J.Q.

and Chen

X.H.

, Multi-criteria decision-making approach based on gray linguistic weighted Bonferroni mean operator, International Transactions in Operational Research (2015). doi: 10.1111/itor.12220

34.

Wang

J.Q.

, Han

Z.Q.

and Zhang

H.Y.

, Multi-criteria group decision-making method based on intuitionistic interval fuzzy information, Group Decision and Negotiation 23(4) (2014), 715–733.

35.

Moshtaghi

, Leckie

, Karunasekera

, Bezdek

J.C.

and Rajasegarar

, Incremental elliptical boundary estimation for anomaly detection in wireless sensor networks, In: 2011 IEEE 11th International Conference on Data Mining, 2011, pp. 467–476.

36.

Tax

D.M.J.

and Duin

R.P.W.

, Data description in subspaces, Proceedings of the International Conference on Pattern Recognition 2 (2000), 672–675.

37.

Wang

J.Q.

, Peng

J.J.

, Zhang

H.Y.

, Liu

and Chen

X.H.

, An uncertain linguistic multi-criteria group decision-making method based on a cloud model, Group Decision and Negotiation 24(1) (2015), 171–192.

38.

Zhang

H.Y.

, Ji

, Wang

and Chen

X.H.

, A neutrosophic normal cloud and its application in decision-making, Cognitive Computation 8(4) (2016), 649–669.

39.

Wang

J.Q.

, Kuang

J.J.

, Wang

and Zhang

H.Y.

, An extended outranking approach to rough stochastic multi-criteria decision-making problems, Cognitive Computation (2016). doi: 10.1007/s12559-016-9417-5

40.

Jiang

J.B.

, Liang

J.R.

, Jiang

and Gu

Z.P.

, Application of trapezium-cloud model in conception division and conception exaltation, Computer Engineering and Design 29(5) (2008), 1235–1240.

41.

Wang

J.Q.

, Wang

, Zhang

H.Y.

and Chen

X.H.

, Atanassov’s interval-valued intuitionistic linguistic multi-criteria group decision-making method based on trapezium cloud model, IEEE Transactions on Fuzzy Systems 23(3) (2015), 338–351.

42.

Lin

W.M.

, Dou

W.C.

, Zhou

Z.J.

and Liu

, A cloud-based framework for home-diagnosis service over big medical data, The Journal of Systems and Software 102 (2014), 1–15.

43.

Chang

T.C.

and Wang

, A self-testing cloud model for multi-criteria group decision making, Engineering Computations 33(6) (2016), 1767–1783.

44.

Zhou

, Wang

J.Q.

and Zhang

H.Y.

, Multi-criteria decision making approaches based on distance measures for linguistic hesitant fuzzy sets, Journal of the Operational Research Society (2016). doi: 10.1057/jors.2016.41

45.

Wang

, Wang

J.Q.

and Zhang

H.Y.

, A likelihood-based TODIM approach based on multi-hesitant fuzzy linguistic information for evaluation in logistics outsourcing, Computers & Industrial Engineering 99 (2016), 287–299.

46.

S.M.

, Wang

and Wang

J.Q.

, An Interval type-2 fuzzy likelihood-based MABAC approach and its application in selecting hotels on a tourism website, International Journal of Fuzzy Systems (2016). doi: 10.1007/s40815-016-0217-6.