Parallel power loads anomalies are processed by a fast-density peak clustering technique that capitalizes on the hybrid strengths of Canopy and K-means algorithms all within Apache Mahout’s distributed machine-learning environment. The study taps into Apache Hadoop’s robust tools for data storage and processing, including HDFS and MapReduce, to effectively manage and analyze big data challenges. The preprocessing phase utilizes Canopy clustering to expedite the initial partitioning of data points, which are subsequently refined by K-means to enhance clustering performance. Experimental results confirm that incorporating the Canopy as an initial step markedly reduces the computational effort to process the vast quantity of parallel power load abnormalities. The Canopy clustering approach, enabled by distributed machine learning through Apache Mahout, is utilized as a preprocessing step within the K-means clustering technique. The hybrid algorithm was implemented to minimise the length of time needed to address the massive scale of the detected parallel power load abnormalities. Data vectors are generated based on the time needed, sequential and parallel candidate feature data are obtained, and the data rate is combined. After classifying the time set using the canopy with the K-means algorithm and the vector representation weighted by factors, the clustering impact is assessed using purity, precision, recall, and value. The results showed that using canopy as a preprocessing step cut the time it proceeds to deal with the significant number of power load abnormalities found in parallel using a fast density peak dataset and the time it proceeds for the k-means algorithm to run. Additionally, tests demonstrate that combining canopy and the K-means algorithm to analyze data performs consistently and dependably on the Hadoop platform and has a clustering result that offers a scalable and effective solution for power system monitoring.
HasanM.K.AhmedM.M.HashimA.H.A.RazzaqueA.IslamS. and PandeyB., A novel artificial intelligence based timing synchronization scheme for smart grid applications, Wirel. Pers. Commun114(2) (Sep. 2020), 1067–1084. doi: 10.1007/s11277-020-07408-w.
2.
AL-JumailiA.H.A.Al MashhadanyY.I.SulaimanR. and AlyasseriZ.A.A., A conceptual and systematics for intelligent power management system-based cloud computing: Prospects, and challenges, Appl. Sci11(21) (Oct. 2021), 9820. doi: 10.3390/APP11219820.
3.
RaoS.N.V.B. et al., Day-ahead load demand forecasting in urban community cluster microgrids using machine learning methods, Energies15(17) (Aug. 2022), 6124. doi: 10.3390/en15176124.
4.
AL-JumailiA.H.A.MuniyandiR.C.HasanM.K.SinghM.J.PawJ.K.S. and AmirM., Advancements in intelligent cloud computing for power optimization and battery management in hybrid renewable energy systems: A comprehensive review, Energy Reports10 (2023), 2206–2227. doi: 10.1016/j.egyr.2023.09.029.
5.
GuoY.WuW.ZhangB. and SunH., An efficient state estimation algorithm considering zero injection constraints, IEEE Trans. Power Syst28(3) (2013), 2651–2659. doi: 10.1109/TPWRS.2012.2232316.
6.
SabirZ.Asif Zahoor RajaM.GuiraoJ.L.G. and ShoaibM., A novel design of fractional Meyer wavelet neural networks with application to the nonlinear singular fractional Lane-Emden systems, Alexandria Eng. J60(2) (2021), 2641–2659. doi: 10.1016/j.aej.2021.01.004.
7.
KimY. and BangH., Introduction to Kalman Filter and Its Applications, vol. 1. IntechOpen London, UK, 2019.
8.
HoggS. and LeschzinerM.A., Computation of highly swirling confined flow with a reynolds stress turbulence model, AIAA J27(1) (1989), 57–63. doi: 10.2514/3.10094.
9.
AmirM., Zaheeruddin and HaqueA., Intelligent based hybrid renewable energy resources forecasting and real time power demand management system for resilient energy systems, Sci. Prog105(4) (Oct. 2022), 003685042211321. doi: 10.1177/00368504221132144.
10.
ChoiY.D.lacovidesH. and LaunderB.E., Numerical computation of turbulent flow in a square-sectioned 180 deg bend, J. Fluids Eng. Trans. ASME111(1) (1989), 59–68. doi: 10.1115/1.3243600.
11.
KoberT.SchifferH.W.DensingM. and PanosE., Global energy perspectives to 2060 – WEC’s World Energy Scenarios 2019, Energy Strateg. Rev31 (2020). doi: 10.1016/j.esr.2020.100523.
12.
HurstW.MontañezC.A.C. and ShoneN., Time-pattern profiling from smart meter data to detect outliers in energy consumption, IoT1(1) (2020), 92–108. doi: 10.3390/iot1010006.
13.
ZanettiM.JamhourE.PellenzM.PennaM.ZambenedettiV. and ChueiriI., A tunable fraud detection system for advanced metering infrastructure using short-lived patterns, IEEE Trans. Smart Grid10(1) (2019), 830–840. doi: 10.1109/TSG.2017.2753738.
14.
YipS.C.TanC.K.TanW.N.GanM.T. and BakarA.H.A., Energy theft and defective meters detection in AMI using linear regression, in: Conference Proceedings – 2017 17th IEEE International Conference on Environment and Electrical Engineering and 2017 1st IEEE Industrial and Commercial Power Systems Europe, EEEIC/I and CPS Europe 2017, 2017. p. 6. doi: 10.1109/EEEIC.2017.7977752.
15.
SinghS.K.BoseR. and JoshiA., Entropy-based electricity theft detection in AMI network, IET Cyber-Physical Syst. Theory Appl3(2) (2018), 99–105. doi: 10.1049/iet-cps.2017.0063.
16.
AlobaidyH.A.H.MandeepJ.S.NordinR.AbdullahN.F.WeiC.G. and SoonM.L.S., Real-World Evaluation of Power Consumption and Performance of NB-IoT in Malaysia, IEEE Internet Things J4662(c) (2021), 1–1. doi: 10.1109/jiot.2021.3131160.
17.
SaifiI.A.HaqueA.AmirM. and Bharath KurukuruV.S., Intelligent Islanding Classification with MLPNN for Hybrid Distributed Energy Generations in Microgrid System, in: 2023 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE), Jan. 2023. pp. 982–987. doi: 10.1109/IITCEE57236.2023.10091089.
18.
Al-JarrahO.Y.Al-HammadiY.YooP.D. and MuhaidatS., Multi-layered clustering for power consumption profiling in smart grids, IEEE Access5 (2017), 18459–18468. doi: 10.1109/ACCESS.2017.2712258.
19.
LiP. et al., High-precision dynamic modeling of two-staged photovoltaic power station clusters, IEEE Trans. Power Syst34(6) (2019), 4393–4407. doi: 10.1109/TPWRS.2019.2915283.
20.
WangY.YangH.XieX.YangX. and ChenG., Real-time subsynchronous control interaction monitoring using improved intrinsic time-scale decomposition, J. Mod. Power Syst. Clean Energy11(3) (May 2023), 816–826. doi: 10.35833/MPCE.2021.000464.
21.
MinH. et al., Toward interpretable anomaly detection for autonomous vehicles with denoising variational transformer, Eng. Appl. Artif. Intell., Jan. 2024, 107601. doi: 10.1016/j.engappai.2023.107601.
22.
ZhaoX.FangY.MinH.WuX.WangW. and TeixeiraR., Potential sources of sensor data anomalies for autonomous vehicles: An overview from road vehicle safety perspective, Expert Syst. Appl.236 (Feb. 2024). doi: 10.1016/j.eswa.2023.121358.
23.
CaoB.ZhangW.WangX.ZhaoJ.GuY. and ZhangY., A memetic algorithm based on two_Arch2 for multi-depot heterogeneous-vehicle capacitated arc routing problem, Swarm Evol. Comput.63 (2021), 100864.
24.
SinghK.AmirM.AhmadF. and RefaatS.S., Enhancement of frequency control for stand-alone multi-microgrids, IEEE Access9 (2021), 79128–79142. doi: 10.1109/ACCESS.2021.3083960.
25.
SinghK.AmirM. and AryaY., Optimal dynamic frequency regulation of renewable energy based hybrid power system utilizing a novel TDF-TIDF controller, Energy Sources, Part A Recover. Util. Environ. Eff44(4) (Dec. 2022), 10733–10754. doi: 10.1080/15567036.2022.2158251.
26.
HasanM.K. et al., Dynamic load modeling for bulk load-using synchrophasors with wide area measurement system for smart grid real-time load monitoring and optimization, Sustain. Energy Technol. Assessments57 (2023), 103190. doi: 10.1016/j.seta.2023.103190.
27.
WangY.ChenQ.KangC. and XiaQ., Clustering of electricity consumption behavior dynamics toward big data applications, IEEE Trans. Smart Grid7(5) (2016), 2437–2447. doi: 10.1109/TSG.2016.2548565.
28.
AghabozorgiS.Seyed ShirkhorshidiA. and Ying WahT., Time-series clustering – A decade review, Inf. Syst53 (2015), 16–38. doi: 10.1016/j.is.2015.04.007.
29.
HassanM.H. and MuniyandiR.C., An improved hybrid technique for energy and delay routing in mobile ad-hoc networks, Int. J. Appl. Eng. Res12(1) (2017), 134–139.
30.
GongC.SuZ.WangP. and YouY., Distributed evidential clustering toward time series with big data issue, Expert Syst. Appl191(August 2021) (2022), 116279. doi: 10.1016/j.eswa.2021.116279.
31.
ShahN.HaqueA.AmirM. and KumarA., Investigation of Renewable Energy Integration Challenges and Condition Monitoring Using Optimized Tree in Three Phase Grid System, in: 2023 7th International Conference on Computing Methodologies and Communication (ICCMC), Feb. 2023. pp. 1582–1588. doi: 10.1109/ICCMC56507.2023.10083636.
32.
AyoubS.HaqueA.AmirM. and KurukuruV.S.B., Intelligent Islanding Classification with Optimal k-Nearest Neighbors Technique for Single Phase Grid Integrated PV System, in: 2022 IEEE 3rd Global Conference for Advancement in Technology (GCAT), Oct. 2022. pp. 1–6. doi: 10.1109/GCAT55367.2022.9972088.
33.
ElkawkagyM. and ElbehH., High performance hadoop distributed file system, Int. J. Networked Distrib. Comput8(3) (2020), 119–123.
34.
AL-JumailiA.H.A.MuniyandiR.C.HasanM.K.PawJ.K.S. and SinghM.J., Big data analytics using cloud computing based frameworks for power management systems: Status, constraints, and future recommendations, Sensors23(6) (2023), 2952. doi: 10.3390/s23062952.
35.
Al-SharqiF.AhmadA.G. and Al-QuranA., Interval-valued neutrosophic soft expert set from real space to complex space, C. Model. Eng. Sci132(1) (2022), 267–293.
36.
OussousA.BenjellounF.-Z.Ait LahcenA. and BelfkihS., Big Data technologies: A survey, J. King Saud Univ. – Comput. Inf. Sci30(4) (2018), 431–448. doi: 10.1016/j.jksuci.2017.06.001.
37.
AnilR. et al., Apache mahout: Machine learning on distributed dataflow systems, J. Mach. Learn. Res21(1) (2020), 4999–5004.
38.
PopD., Machine Learning and Cloud Computing: Survey of Distributed and SaaS Solutions, arXiv Prepr. arXiv1603. 08767, 2016, [Online]. Available: http://arxiv.org/abs/1603.08767.
39.
PalaniswamiM.RaoA.S.KumarD.RathoreP. and RajasegararS., The role of visual assessment of clusters for big data analysis: From real-world internet of things, IEEE Syst. Man, Cybern. Mag6(4) (2020), 45–53. doi: 10.1109/msmc.2019.2961160.
40.
XiaD.NingF. and HeW., Research on Parallel Adaptive Canopy-K-Means Clustering Algorithm for Big Data Mining Based on Cloud Platform, J. Grid Comput18(2) (2020), 263–273. doi: 10.1007/s10723-019-09504-z.
41.
YuanC. and YangH., Research on K-Value Selection Method of K-Means Clustering Algorithm, J2(2) (2019), 226–235. doi: 10.3390/j2020016.
42.
TarekegnA.N.MichalakK. and GiacobiniM., Cross-validation approach to evaluate clustering algorithms: An experimental study using multi-label datasets, SN Comput. Sci1(5) (2020), 1–9. doi: 10.1007/s42979-020-00283-z.
43.
SinghK.DahiyaM.GroverA.AdlakhaR. and AmirM., An effective cascade control strategy for frequency regulation of renewable energy based hybrid power system with energy storage system, J. Energy Storage68 (Sep. 2023), 107804. doi: 10.1016/j.est.2023.107804.
44.
AnsariM.Y.AhmadA.KhanS.S.BhushanG. and Mainuddin, Spatiotemporal clustering: A review, Artif. Intell. Rev53(4) (2020), 2381–2423. doi: 10.1007/s10462-019-09736-1.
45.
TaamnehS.QawasmehA. and AljammalA.H., Parallel and fault-tolerant k-means clustering based on the actor model, Multiagent Grid Syst16(4) (2020), 379–396. doi: 10.3233/MGS-200336.
46.
CapóM.PérezA. and LozanoJ.A., An efficient K-means clustering algorithm for tall data, Data Min. Knowl. Discov34(3) (2020), 776–811. doi: 10.1007/s10618-020-00678-9.
47.
MaroosiA.MuniyandiR.C.SundararajanE. and ZinA.M., Parallel and distributed computing models on a graphics processing unit to accelerate simulation of membrane systems, Simul. Model. Pract. Theory47 (2014), 60–78. doi: 10.1016/j.simpat.2014.05.005.
48.
MaroosiA. and MuniyandiR.C., Accelerated execution of P systems with active membranes to solve the N-queens problem, Theor. Comput. Sci551(C) (2014), 39–54. doi: 10.1016/j.tcs.2014.05.004.
49.
FengZ.HuangJ.TangW.H. and ShahidehpourM., Data mining for abnormal power consumption pattern detection based on local matrix reconstruction, Int. J. Electr. Power Energy Syst123(March) (2020), 106315. doi: 10.1016/j.ijepes.2020.106315.
