Sage Journals: Discover world-class research

Abstract

With the rapid development of smart grid technology, a large amount of multi-source heterogeneous data has been generated in the power system, and its effective utilization is crucial for the optimization operation, demand prediction, and anomaly detection of the power system. However, the fusion processing of multi-source heterogeneous data faces many challenges, such as inconsistent data format, granularity, and quality, and direct fusion can easily lead to information redundancy and contradictions. A multi-source heterogeneous data fusion technology based on big data mining has been proposed to address the above issues. This method combines the advantages of convolutional neural networks and gated recurrent units to automatically extract features from image and sequence data and handle long-term dependency issues in time series data. Meanwhile, the K-means clustering algorithm is used to preprocess the data and train a specialized ConvGRU model. The results showed that in short-term load forecasting and abnormal electricity consumption behaviour detection tasks, the accuracy of this method reached 96.3% and 98.7%, respectively, with AUC values of 0.994 and 0.996. Compared to models that use only CNN or GRU, the performance is significantly improved. This method effectively solves the problem of integrating and processing multi-source heterogeneous power data, improves the accuracy and efficiency of power system data analysis, and provides strong support for the optimized operation of smart grids.

Keywords

convolutional neural network customer data short-term prediction abnormal electricity use behaviour clustering

Get full access to this article

View all access options for this article.

References

Bohm

Sawan

. Neutronics calculations to support the fusion evaluated nuclear data library (FENDL). Fusion Sci Technol 2021; 77(7-8): 813–828.

Stan

. Statistical and machine-learning data mining: techniques for better predictive modeling and analysis of big data. Technometrics 2022; 64(1): 145–148.

Zhang

Jia

, et al. Fast 3D-HEVC inter coding using data mining and machine learning. IET Image Process 2022; 16(11): 3067–3084.

Hung

Kang

, et al. Chemistry-encoded convolutional neural networks for predicting gaseous adsorption in porous materials. J Phys Chem C 2022; 126(5): 2813–2822.

Meng

Zhang

Wang

, et al. Data mining to atmospheric corrosion process based on evidence fusion. Materials 2021; 14(22): 6954.

Schaap

Cardozo

Patel

, et al. Image-based automated psoriasis area severity index scoring by convolutional neural networks. J Eur Acad Dermatol 2022; 36(1): 68–75.

Venkatesh

Sugumaran

. A combined approach of convolutional neural networks and machine learning for visual fault classification in photovoltaic modules. P I Mech Eng O-J Ris 2022; 236(1): 148–159.

Dreier

Sorensen

Lund-Hansen

, et al. Hyperspectral imaging for classification of bulk grain samples with deep convolutional neural networks. Near Infrared Spec 2022; 30(3): 107–121.

Jeong

Almubarak

Tsingas

. Quality control for the geophone reorientation of ocean bottom seismic data using k-means clustering. Geophys Prospect 2021; 69(7): 1487–1502.

10.

Jana

Gupta

. K-means clustering-based detection for WDM PD-NOMA system with mixed fiber types. Opt Eng 2021; 60(8): 086109.

11.

Feldman

Schmidt

Sohler

. Turning big data into tiny data: constant-size coresets for k-means, PCA, and projective clustering. SIAM J Comput 2020; 49(3): 601–657.

12.

Guo

Cheng

Wen

, et al. High-frequency small angle measurement using a dynamic autocollimator based on an improved k-means algorithm. Opt Eng 2018; 57(6): 064109.

13.

Al Hasib

Cebrian

Natvig

. A vectorized k-means algorithm for compressed datasets: design and experimental analysis. J Supercomput 2018; 74(6): 2705–2728.

14.

Jana

Shukla

Gupta

. K-Means algorithm-based detection for wavelength division multiplexed OOK PD-NOMA system over turbulent optical channel. Opt Eng 2022; 61(3): 036111.

15.

Tighe

, et al. Illumination compensation model with k-means algorithm for detection of pavement surface cracks with shadow. J Comput Civ Eng 2020; 34(1): 04019049.

16.

Yuan

Virupakshappa

Oruklu

. FPGA implementation of an ultrasonic flaw detection algorithm based on convolutional neural networks. J Signal Process Syst 2022; 94(12): 1447–1457.

17.

Nazari

Kluge

Apostolova

, et al. Explainable AI to improve acceptance of convolutional neural networks for automatic classification of dopamine transporter SPECT in the diagnosis of clinically uncertain parkinsonian syndromes. Eur J Nucl Med Mol Imag 2022; 49(4): 1176–1186.

18.

Trevino-Sanchez

Alarcon-Aquino

. Hybrid pooling with wavelets for convolutional neural networks. J Intell Fuzzy Syst 2022; 42(5): 4327–4336.

19.

Friggstad

Rezapour

Salavatipour

. Local search yields a PTAS for k-means in doubling metrics. SIAM J Comput 2019; 48(2): 452–480.

20.

Zhang

. Research on control strategy of multi-source data fusion solar intelligent vehicle based on image recognition. Int J Low Carbon Technol 2021; 16(4): 1363–1370.

21.

Shi

Ren

Yan

, et al. Improving soil moisture retrieval from GNSS-interferometric reflectometry: parameters optimization and data fusion via neural network. Int J Rem Sens 2021; 42(23): 9076–9099.

22.

De Leonardo

Del Fabio

Latini

, et al. SAR data fusion and a novel joint use of neural networks for coastline extraction. Int J Rem Sens 2021; 42(22): 8734–8759.

23.

Zhang

Chen

Wulamu

, et al. TransG-net: transformer and graph neural network based multi-modal data fusion network for molecular properties prediction. Appl Intell 2023; 53(12): 16077–16088.

24.

Zhang

Catalucci

Thompson

, et al. Applications of data fusion in optical coordinate metrology: a review. Int J Adv Manuf Technol 2023; 124(5-6): 1341–1356.

Multi-source heterogeneous data fusion technology for electric power based on big data mining

Abstract

Keywords

Get full access to this article

References