Imbalanced data classification using improved synthetic minority over-sampling technique

Abstract

In data mining, deep learning and machine learning models face class imbalance problems, which result in a lower detection rate for minority class samples. An improved Synthetic Minority Over-sampling Technique (SMOTE) is introduced for effective imbalanced data classification. After collecting the raw data from PIMA, Yeast, E.coli, and Breast cancer Wisconsin databases, the pre-processing is performed using min-max normalization, cleaning, integration, and data transformation techniques to achieve data with better uniqueness, consistency, completeness and validity. An improved SMOTE algorithm is applied to the pre-processed data for proper data distribution, and then the properly distributed data is fed to the machine learning classifiers: Support Vector Machine (SVM), Random Forest, and Decision Tree for data classification. Experimental examination confirmed that the improved SMOTE algorithm with random forest attained significant classification results with Area under Curve (AUC) of 94.30%, 91%, 96.40%, and 99.40% on the PIMA, Yeast, E.coli, and Breast cancer Wisconsin databases.

Keywords

Data cleaning imbalanced data min max normalization random forest synthetic minority oversampling technique transformation

Get full access to this article

View all access options for this article.

References

Yao

and Ge

, Scalable semisupervised GMM for big data quality prediction in multimode processes, IEEE Transactions on Industrial Electronics 66(5) (2019), 3681–3692. doi: 10.1109/TIE.2018.2856200.

Yao

and Ge

, Distributed parallel deep learning of hierarchical extreme learning machine for multimode quality prediction with big process data, Engineering Applications of Artificial Intelligence 81 (2019), 450–465. doi: 10.1016/j.engappai.2019.03.011.

Ed-daoudy

and Maalmi

, A new Internet of Things architecture for real-time prediction of various diseases using machine learning on big data environment, Journal of Big Data 6 (2019), 104. doi: 10.1186/s40537-019-0271-7.

Rashid

Singh

Goyal

Parah

S.A.

and Wani

A.R.

, Big data based hybrid machine learning model for improving performance of medical Internet of Things data in healthcare systems, in: Healthcare Paradigms in the Internet of Things Ecosystem Balas

V.E.

Pal

, eds., Academic Press, 2021, Chapter-3, pp. 47–62. doi: 10.1016/B978-0-12-819664-9.00003-X.

Hassib

E.M.

El-Desouky

A.I.

El-Kenawy

E.-S.M.

and El-Ghamrawy

S.M.

, An imbalanced big data mining framework for improving optimization algorithms performance, IEEE Access 7 (2019), 170774–170795. doi: 10.1109/ACCESS.2019.2955983.

Zhou

Sun

Liu

Zhou

and Zhou

, A big data mining approach of PSO-based BP neural network for financial risk management with IoT, IEEE Access 7 (2019), 154035–154043. doi: 10.1109/ACCESS.2019.2948949.

Zhu

and Feng

, Research on big data mining based on improved parallel collaborative filtering algorithm, Cluster Computing 22(2) (2019), 3595–3604. doi: 10.1007/s10586-018-2209-9.

Kim

J.-C.

and Chung

, Associative feature information extraction using text mining from health big data, Wireless Personal Communications 105(2) (2019), 691–707. doi: 10.1007/s11277-018-5722-5.

Hadi

M.S.

Lawey

A.Q.

El-Gorashi

T.E.H.

and Elmirghani

J.M.H.

, Patient-centric cellular networks optimization using big data analytics, IEEE Access 7 (2019), 49279–49296. doi: 10.1109/ACCESS.2019.2910224.

10.

Gupta

and Jivani

, A cluster-based under-sampling solution for handling imbalanced data, International Journal on Emerging Technologies 10(4) (2019), 160–170.

11.

Ikotun

A.M.

Almutari

M.S.

and Ezugwu

A.E.

, K-means-based nature-inspired metaheuristic algorithms for automatic data clustering problems: Recent advances and future directions, Applied Sciences 11(23) (2021), 11246. doi: 10.3390/app112311246.

12.

Gong

Wang

and You

, Evidential instance selection for K-nearest neighbor classification of big data, International Journal of Approximate Reasoning 138 (2021), 123–144. doi: 10.1016/j.ijar.2021.08.006.

13.

Sun

Liu

and Zhang

, An adaptive density peaks clustering method with Fisher linear discriminant, IEEE Access 7 (2019), 72936–72955. doi: 10.1109/ACCESS.2019.2918952.

14.

Zhang

Bouras

and Wang

, Hybrid missing value imputation algorithms using fuzzy c-means and vaguely quantified rough set, IEEE Transactions on Fuzzy Systems 30(5) (2022), 1396–1408. doi: 10.1109/TFUZZ.2021.3058643.

15.

Farajian

and Adibi

, Minority manifold regularization by stacked auto-encoder for imbalanced learning, Expert Systems with Applications 169 (2021), 114317. doi: 10.1016/j.eswa.2020.114317.

16.

Hassib

E.M.

El-Desouky

A.I.

Labib

L.M.

and El-kenawy

E.-S.M.

, WOA+BRNN: An imbalanced big data classification framework using Whale optimization and deep neural network, Soft Computing 24(8) (2020), 5573–5592. doi: 10.1007/s00500-019-03901-y.

17.

Lakshmanaprabu

S.K.

Shankar

Ilayaraja

Nasir

A.W.

Vijayakumar

and Chilamkurti

, Random forest for big data classification in the internet of things using optimal features, International Journal of Machine Learning and Cybernetics 10(10) (2019), 2609–2618. doi: 10.1007/s13042-018-00916-z.

18.

Elkano

Sanz

J.A.

Barrenechea

Bustince

and Galar

, CFM-BD: A distributed rule induction algorithm for building Compact Fuzzy Models in Big Data classification problems, IEEE Transactions on Fuzzy Systems 28(1) (2020), 163–177. doi: 10.1109/TFUZZ.2019.2900856.

19.

Manogaran

Vijayakumar

Varatharajan

Kumar

P.M.

Sundarasekar

and Hsu

C.-H.

, Machine learning based big data processing framework for cancer diagnosis using hidden Markov model and GM clustering, Wireless Personal Communications 102(3) (2018), 2099–2116. doi: 10.1007/s11277-017-5044-z.

20.

El-Hasnony

I.M.

Barakat

S.I.

Elhoseny

and Mostafa

R.R.

, Improved feature selection model for big data analytics, IEEE Access 8 (2020), 66989–67004. doi: 10.1109/ACCESS.2020.2986232.

21.

Venkatasalam

Rajendran

and Thangavel

, Improving the accuracy of feature selection in big data mining using accelerated flower pollination (AFP) Algorithm, Journal of Medical Systems 43(4) (2019), 96. doi: 10.1007/s10916-019-1200-1.

22.

Zhang

Hao

and Fan

, Optimization and improvement of data mining algorithm based on efficient incremental kernel fuzzy clustering for large data, Cluster Computing 22(2) (2019), 3001–3010. doi: 10.1007/s10586-018-1767-1.

23.

Ilango

S.S.

Vimal

Kaliappan

and Subbulakshmi

, Optimization using artificial bee colony based clustering approach for big data, Cluster Computing 22(5) (2019), 12169–12177. doi: 10.1007/s10586-017-1571-3.

24.

Ahmad

Khan

Paul

Din

Rathore

M.M.

Jeon

and Choi

G.S.

, Toward modeling and optimization of features selection in Big Data based social Internet of Things, Future Generation Computer Systems 82 (2018), 715–726. doi: 10.1016/j.future.2017.09.028.

25.

Kushwaha

and Pant

, Link based BPSO for feature selection in big data text clustering, Future Generation Computer Systems 82 (2018), 190–199. doi: 10.1016/j.future.2017.12.005.

26.

Livieris

I.E.

and Pintelas

, An improved weight-constrained neural network training algorithm, Neural Computing and Applications 32(9) (2020), 4177–4185. doi: 10.1007/s00521-019-04342-2.

27.

Aziz

M.F.A.

Mostafa

S.A.

Foozy

C.F.M.

Mohammed

M.A.

Elhoseny

and Abualkishik

A.Z.

, Integrating Elman recurrent neural network with particle swarm optimization algorithms for an improved hybrid training of multidisciplinary datasets, Expert Systems with Applications 183 (2021), 115441. doi: 10.1016/j.eswa.2021.115441.

28.

Zheng

Chen

Zhou

and Yang

, UFFDFR: Undersampling framework with denoising, fuzzy c-means clustering, and representative sample selection for imbalanced data classification, Information Sciences 576 (2021), 658–680. doi: 10.1016/j.ins.2021.07.053.

29.

Rustogi

and Prasad

, Swift imbalance data classification using SMOTE and extreme learning machine, in: International Conference on Computational Intelligence in Data Science (ICCIDS), Chennai, India, 21–23 February, IEEE, 2019, pp. 1–6. doi: 10.1109/ICCIDS.2019.8862112.

30.

J.-H.

Tian

Liu

W.-h.

and Lou

J.-g.

, Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning, International Journal of Machine Learning and Cybernetics 11(9) (2020), 2007–2020. doi: 10.1007/s13042-020-01090-x.

31.

Elyan

Moreno-Garcia

C.F.

and Jayne

, CDSMOTE: Class decomposition and synthetic minority class oversampling technique for imbalanced-data classification, Neural Computing and Applications 33(7) (2021), 2839–2851. doi: 10.1007/s00521-020-05130-z.

32.

Ibrahim

M.H.

, ODBOT: Outlier detection-based oversampling technique for imbalanced datasets learning, Neural Computing and Applications 33(22) (2021), 15781–15806. doi: 10.1007/s00521-021-06198-x.

33.

Rubbo

and Silva

L.A.

, Filtering-based instance selection method for overlapping problem in imbalanced datasets, J – Multidisciplinary Scientific Journal 4(3) (2021), 308–327. doi: 10.3390/j4030024.

34.

Kumar

Dabas

and Hooda

, Text classification algorithms for mining unstructured data: A SWOT analysis, International Journal of Information Technology 12(4) (2020), 1159–1169. doi: 10.1007/s41870-017-0072-1.

35.

Subramaniyaswamy

Vijayakumar

Logesh

and Indragandhi

, Unstructured data analysis on big data using map reduce, Procedia Computer Science 50 (2015), 456–465. doi: 10.1016/j.procs.2015.04.015.

36.

Madhusudhanan

Jaganathan

and Jayashree

L.S.

, Incremental learning for classification of unstructured data using extreme learning machine, Algorithms 11(10) (2018), 158. doi: 10.3390/a11100158.

37.

Ashok

Kadhar

G.M.

Elayaraja

and Vadivel

, Fuzzy based clustering method on yeast dataset with different fuzzification methods, in: Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), Tiruchengode, India, 04–06 July, IEEE, 2013, pp. 1–6. doi: 10.1109/ICCCNT.2013.6726574.

38.

Dubey

A.K.

Gupta

and Jain

, Analysis of k-means clustering approach on the breast cancer Wisconsin dataset, International Journal of Computer Assisted Radiology and Surgery 11(11) (2016), 2033–2047. doi: 10.1007/s11548-016-1437-9.

39.

Chai

Deng

Yang

and Ling

C.X.

, Test-cost sensitive naive Bayes classification, in: Fourth IEEE International Conference on Data Mining (ICDM’04), Brighton, UK, 01–04 November, IEEE, 2004, pp. 51–58. doi: 10.1109/ICDM.2004.10092.

40.

Gajera

, Shubham Gupta

and Jana

P.K.

, An effective multi-objective task scheduling algorithm using min-max normalization in cloud computing, in: 2nd International Conference on Applied and Theoretical Computing and Communication Technology (iCATccT), Bangalore, India, 21–23 July, IEEE, 2016, pp. 812–816. doi: 10.1109/ICATCCT.2016.7912111.

41.

Sun

Fujita

and Ai

, Class-imbalanced dynamic financial distress prediction based on Adaboost-SVM ensemble combined with SMOTE and time weighting, Information Fusion 54 (2020), 128–144. doi: 10.1016/j.inffus.2019.07.006.

42.

Speiser

J.L.

Miller

M.E.

Tooze

and Ip

, A comparison of random forest variable selection methods for classification prediction modeling, Expert Systems with Applications 134 (2019), 93–101. doi: 10.1016/j.eswa.2019.05.028.