Research on association rules of course grades based on parallel FP-Growth algorithm

Abstract

With the rapid growth of massive data in all walks of life, massive data faces enormous challenges such as storage capacity and computing power. In Chinese universities, traditional data analysis of student course cannot meet the growing demand for increasing data size and real-time computation of big data. In this paper, a parallel FP-Growth algorithm based on split is proposed. The established FP-Tree is split into blocks, and the split FP-Trees are equally divided into different nodes. The monitoring point is set up to monitor the operation of other nodes, dynamically migrate tasks and maintain load balancing. The experiment proves that each node has good load balancing with the given support degree, and the improved algorithm has better running performance than the classic FP-Growth algorithm in parallel processing. Finally, the parallel FP-Growth algorithm based on split is implemented on Hadoop to mine association rules between course grades. The mining process includes data preprocessing, mining results and analysis. The association rules between course grades provide suggestions for the way students learn and the way teachers teach.

Keywords

Association rules Hadoop MapReduce parallel FP-Growth algorithm

Get full access to this article

View all access options for this article.

References

Han

J.W.

et al., Mining frequent patterns without candidate generation: A frequent-pattern tree Approach, Data Mining and Knowledge Discovery (2004), 53–87.

J.H.

et al., Optimization and implementation of parallel FP-Growth algorithm based on Spark, Journal of Computer Applications 11 (2018), 3069–3074.

Y.K.

et al., Improved FP-Growth algorithm and its distributed parallel implementation, Journal of Harbin University of Science and Technology 2 (2016), 20–27.

Jiang

et al., A parallel FP-growth algorithm on World Ocean Atlas data with multi-core CPU, Journal of Supercomputing 75(2) (2019), 732–745.

Choi

S.Y.

and Chung

, Knowledge process of health big data using MapReduce-based associative mining, Personal and Ubiquitous Computing (2019), 1–11.

Yekkehkhany

Hojjati

and Hajiesmaili

M.H.

, GB-PANDAS: Throughput and heavy-traffic optimality analysis for affinity scheduling, ACM Sigmetrics Performance Evaluation Review 45(3) (2018), 2–14.

Liao

et al., Energy-Efficient Algorithms for Distributed File System HDFS, Chinese Journal of Computers 36(5) (2013), 1047–1064.

Wei

et al., Incremental FP-Growth mining strategy for dynamic threshold value and database based on MapReduce, IEEE International Conference on Computer Supported Cooperative Work in Design (2014), 271–276.

M.B.

Ding

and Yi

, Data analysis of tyre quality based on improved FP-Growth algorithm, China Mechanical Engineering 30(2) (2019), 244–251.

10.

Zhang

Z.Y.

et al., A novel distributed parallel FP-Growth algorithm based on Hadoop framework, Hebei Journal of Industrial Science and Technology (2016), 169–177.

11.

Cho

et al., Analysis of anti-cancer cytokines by Apriori algorithm, decision tree, and SVM, International Conference on Big Data and Smart Computing (2015), 232–237.

12.

and Li

L.J.

, Design and parallel implementation of minimum spanning tree based K-means algorithm, Journal of Nanjing University of Posts and Telecommunications (Natural Science Edition) (2017), 81–86.

13.

Feng

et al., FDTD parallel algorithm based on distributed platform, Journal of Beijing University of Aeronautics and Astronautics 42(9) (2016), 1874–1883.

14.

Jia

K.K.

and Liu

H.B.

, An improved FP-Growth algorithm based on SOM partition, Abstracts of the Third International Conference of Pioneering Computer Scientists 727 (2017), 62–64.

15.

Khader

Lashier

and Yoon

S.W.

, Pharmacy robotic dispensing and planogram analysis using association rule mining with prescription data, Expert Systems with Applications 57 (2016), 296–310.

16.

Cheng

and Gong

, Research on audit log association rule mining based on improved Apriori algorithm, IEEE International Conference on Big Data Analysis (2016), 1–7.

17.

Chang

H.Y.

et al., A hybrid algorithm for frequent pattern mining using MapReduce framework, 2015 1st International Conference on Computational Intelligence Theory Systems and Applications (CCITSA) (2015), pp. 19–22.

18.

Wang

and Zhao

J.H.

, Virtual network embedding algorithm for load balance with various requests, Chinese Journal of Electronics 23(2) (2014), 382–387.

19.

Liu

Wang

T.M.

and Chen

J.X.

, Load balancing strategy for cloud computing based on dynamic replica technology, Journal of Computational Methods in Sciences and Engineering 19(4) (2019), 1–11.

20.

Liu

Zhang

D.W.

and Gao

Z.J.

, A novel load balancing strategy based on node load comprehensive measuring under cloud computing environment, Journal of Computational Methods in Sciences and Engineering 19(1) (2019), 3–9.

21.

Mao

, The connection rule excavation related algorithm study, Southwest JiaoTong University (2009), pp. 8–9.

22.

Xie

Q.M.

Yekkehkhany

and Lu

, Scheduling with multi-level data locality: Throughput and heavy-traffic optimality, In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications (2016), pp. 1–9.

23.

Shi

and Qian

X.Z.

, Research and implementation of parallel FP-Growth algorithm based on Hadoop, Microelectronics & Computer (2015), 150–154.

24.

Borah

and Nath

, Identifying risk factors for adverse diseases using dynamic rare association rule mining, Expert Systems with Applications 113 (2018), 233–263.