An improved hybrid structure learning strategy for Bayesian networks based on ensemble learning

Abstract

In the application of Bayesian networks to solve practical problems, it is likely to encounter the situation that the data set is expensive and difficult to obtain in large quantities and the small data set is easy to cause the inaccuracy of Bayesian network (BN) scoring functions, which affects the BN optimization results. Therefore, how to better learn Bayesian network structures under a small data set is an important problem we need to pay attention to and solve. This paper introduces the idea of parallel ensemble learning and proposes a new hybrid Bayesian network structure learning algorithm. The algorithm adopts the elite-based structure learner using genetic algorithm (ESL-GA) as the base learner. Firstly, the adjacency matrices of the network structures learned by ESL-GA are weighted and averaged. Then, according to the preset threshold, the edges between variables with weak dependence are filtered to obtain a fusion matrix. Finally, the fusion matrix is modified as the adjacency matrix of the integrated Bayesian network so as to obtain the final Bayesian network structure. Comparative experiments on the standard Bayesian network data sets show that the accuracy and reliability of the proposed algorithm are significantly better than other algorithms.

Keywords

Bayesian networks structural learning genetic algorithm ensemble learning

Get full access to this article

View all access options for this article.

References

Pearl

, Fusion, propagation, and structuring in belief networks, Artificial Intelligence 29(3) (1986), 241–288.

Hitzel

et al., Oxidized phospholipids regulate amino acid metabolism through MTHFD2 to facilitate nucleotide release in endothelial cells, Nature Communications 9 (2018).

Huang

et al., A sparse structure learning algorithm for Gaussian Bayesian Network identification from high-dimensional data, IEEE Trans Pattern Anal Mach Intell 35(6) (2013), 1328–1342.

Tsamardinos

et al., The max-min hill-climbing Bayesian network structure learning algorithm, Machine Learning 65(1) (2006), 31–78.

Gasse

et al., A hybrid algorithm for Bayesian network structure learning with application to multi-label learning, Expert Systems with Applications 41(15) (2014), 6755–6772.

Sun

X.P.

et al., Hybrid Optimization Algorithm for Bayesian Network Structure Learning, Information 10(10) (2019).

Akay

and Karaboga

, A modified Artificial Bee Colony algorithm for real-parameter optimization, Information Sciences 192 (2012), 120–142.

Katoch

et al., A review on genetic algorithm: Past, present, and future, Multimed Tools Appl 80(5) (2021), 8091–8126.

Larranaga

et al., Learning Bayesian network structures by searching for the best ordering with genetic algorithms, Ieee Transactions on Systems Man and Cybernetics Part a-Systems and Humans 26(4) (1996), 487–493.

10.

Larranaga

et al., Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters, Ieee Transactions on Pattern Analysis and Machine Intelligence 18(9) (1996), 912–926.

11.

Lee

et al., Structure learning of Bayesian networks using dual genetic algorithm, Ieice Transactions on Information and Systems E91d(1) (2008), 32–43.

12.

Lee

et al., A New Genetic Approach for Structure Learning of Bayesian Networks: Matrix Genetic Algorithm, International Journal of Control Automation and Systems 8(2) (2010), 398–407.

13.

Guo

et al., Epi-GTBN: An approach of epistasis mining based on genetic Tabu algorithm and Bayesian network, BMC Bioinformatics 20(1) (2019), 444.

14.

Dai

J.G.

et al., An improved evolutionary approach-based hybrid algorithm for Bayesian network structure learning in dynamic constrained search space, Neural Computing & Applications 32(5) (2020), 1413–1434.

15.

Jose

et al., Bayesian Network Structure Learning Using Case-Injected Genetic Algorithms, in: 2020 Ieee 32nd International Conference on Tools with Artificial Intelligence (Ictai), 2020, pp. 572–579.

16.

Contaldi

et al., Bayesian network hybrid learning using an elite-guided genetic algorithm, Artificial Intelligence Review 52(1) (2019), 245–272.

17.

Cooper

G.F.

and Herskovits

, A Bayesian Method for the Induction of Probabilistic Networks from Data, Machine Learning 9(4) (1992), 309–347.

18.

Bouchaala

et al., Improving algorithms for structure learning in Bayesian Networks using a new implicit score, Expert Systems with Applications 37(7) (2010), 5470–5475.

19.

Daly

et al., Learning Bayesian networks: Approaches and issues, Knowledge Engineering Review 26(2) (2011), 99–157.

20.

Rissanen

, A universal prior for integers and estimation by minimum description length, Annals of Statistics 11(2) (1983), 416–431.

21.

Heckerman

et al., Learning Bayesian Networks – the Combination of Knowledge and Statistical-Data, Machine Learning 20(3) (1995), 197–243.

22.

Wang

M.X.

et al., Improved Parameter Uniform Priors in Bayesian Network Structure Learning, in: 2018 4th International Conference on Environmental Science and Material Application, vol. 252, 2019.

23.

Lam

A.Y.S.

and Li

V.O.K.

, Generalization of the No-Free-Lunch Theorem, in: 2009 Ieee International Conference on Systems, Man and Cybernetics (Smc 2009), Vols 1-9, 2009, pp. 4322–4328.

24.

Ren

et al., Ensemble Classification and Regression-Recent Developments, Applications and Future Directions, Ieee Computational Intelligence Magazine 11(1) (2016), 41–53.

25.

Vafaee

et al., Balancing the Exploration and Exploitation in an Adaptive Diversity Guided Genetic Algorithm, 2014 Ieee Congress on Evolutionary Computation (Cec), 2014, 2570–2577.

26.

Vafaee

, Learning the Structure of Large-scale Bayesian Networks using Genetic Algorithm, in: Gecco’14: Proceedings of the 2014 Genetic and Evolutionary Computation Conference, 2014, pp. 855–862.

27.

Contaldi

et al., The Role of Crossover Operator in Bayesian Network Structure Learning Performance: a Comprehensive Comparative Study and New Insights, in: Proceedings of the 2017 Genetic and Evolutionary Computation Conference (Gecco’17), 2017, pp. 769–776.