Since the traditional probabilistic neural network (PNN) cannot systematically solve the difficulty of estimating probability function and the high space complexity, this paper introduces backpropagation (BP) algorithm into the classical PNN. By designing appropriate error function and BP algorithm based on the steepest descent, an improved BP-PNN is presented, with its algorithm and effectiveness deduced. Three synthetic datasets and ten benchmark problems have been tested, compared with Probabilistic Neural Networks (PNN), Multi-Layered Perceptron (MLP) and Support Vector Machine (SVM). The results prove that (1) the accuracy of classification of BP-PNN is much higher than PNN, and it has a significant advantage compared with MLP and SVM; (2) BP-PNN has strong capacity to identify the importance of input indicators; (3) BP-PNN is a new pattern classification method to estimate the probabilistic function, reduce the space complexity and identify the importance of the indicators.
SpechtD.F., Probabilistic neural networks for classification, or associative memory, in: IEEE International Conference on Neural Networks, San Diego, 1988, pp. 525–535.
SpechtD.F. and ShapiroP.D., Generalization accuracy of probabilistic neural networks compared with back-propagation networks, in: International Joint Conference on Neural Networks, Seattle, Vol. 1, 1991, pp. 887–892.
4.
SpechtD.F., Enhancements to probabilistic neural networks, in: International Joint Conference on Neural Networks, Baltimore, Vol. 1, 1992, pp. 761–768.
5.
AdeliH. and PanakkatA., A probabilistic neural network for earthquake magnitude prediction, Neural Networks22(7) (2009), 1018–1024.
6.
SongT., JamshidiM.M., LeeR.R. and HuangM., A modified probabilistic neural network for partial volume segmentation in brain MR image, in: IEEE Transactions on Neural Networks18(5) (2007), 1424–1432.
7.
TangM.Z., YangC.H. and GuiW.H., Cost-sensitive probabilistic neural network with its application in fault diagnosis, Control and Decision25(7) (2010), 1074–1079.
8.
MehdiK. and MehdiB., Hybridization of the probabilistic neural networks with feed-forward neural networks for forecasting, Engineering Applications of Artificial Intelligence (25) (2012), 1277–1288.
9.
PangS.L., Study on credit scoring model and forecasting based on probabilistic neural network, Systems Engineering-Theory & Practice25(5) (2005), 43–48.
10.
JinY.P. and LiB.L., Lithology identification based on genetic optimized radial basis probabilistic neural network, Journal of Computer Applications33(2) (2013), 353–356.
11.
SavchenkoA.V., Probabilistic neural network with homogeneity testing in recognition of discrete patterns set, Neural Networks (46) (2013), 227–241.
12.
JamesO.B., Statistical decision theory and bayesian analysis, Spring-Verlag, New York, 1985.
13.
ThierryD., A K-nearest neighbor classification rule based on Dempster-Shafer theory, Studies in Fuzziness and Soft Computing21(9) (2008), 737–760.
14.
YangZ.R., MarjorieB.P. and HarlanD.P., Probabilistic neural networks in bankruptcy prediction, Journal of Business Research44(2) (1999), 67–74.
15.
MichaelR.B. and JayD., Constructive training of probabilistic neural networks, Neurocomputing19(1-3) (1998), 167–183.
16.
YuS.N. and ChenY.H., Electrocardiogram beat classification based on wavelet transformation and probabilistic neural network, Pattern Recognition Letters28(10) (2007), 1142–1150.
17.
MishraS., BhendeC.N. and PanigrahiB.K., Detection and classification of power quality disturbances using S-transform and probabilistic neural network, in: IEEE Transactions on Power Delivery23(1) (2008), 280–287.
18.
RamakrishnanS. and IbrahiemM.M., Comparative study between traditional and modified probabilistic neural networks, Telecommunication Systems40(1-2) (2010), 67–74.
19.
XingJ. and XiaoD.Y., PCA-based probability neural network structure optimization, Journal of Tsinghua University (Science and Technology)48(1) (2008), 141–144.
20.
HuangD.S. and ZhaoW.B., Determining the centers of radial basis probabilistic neural networks by recursive orthogonal least square algorithms, Applied Mathematics and Computation162(1) (2005), 461–473.
21.
LeeD.C., KimB.J. and YunJ.M., A combined method for improving backpropagation algorithm, in: IEEE International Conference on Systems, Man, and Cybernetics, Charlottesville, Vol. 1-31991, pp. 1581–1586.
22.
IgelC. and HuskenM., Empirical evaluation of the improved Rprop learning algorithms, Neurocomputing50(1) (2003), 105–123.
23.
SammyS., YangS.S. and LeeC.M., Improving the back-propagation algorithm Using Evolutionary Strategy, IEEE Transactions on Circuits and Systems II-Express Briefs52(2) (2007), 171–175in.
24.
JoseP., LeonardoT. and PatriciaM., Hybrid backpropagation training with evolutionary strategies, Soft Computing18(8) (2014), 1603–1614.
25.
KirkpatrickS., GelattC.D. and VecchiM.P., Optimization by simulated annealing, Science220(5) (1983), 671–680.
26.
ChowdhuryP.R., SinghY.P. and ChansarkarR.A., Dynamic tunneling technique for efficient training of multilayer perceptions, IEEE Transaction on Neural Networks10(1) (1999), 48–55in.
27.
YamJ.Y.F. and ChowT.W.S., A weight initialization method for improving training speed in feedforward neural network, Neurocomputing30(1) (2000), 219–232.
28.
RahmanA. and JamshidM., Bottom hole pressure estimation using evolved neural networks by real coded ant colony optimization and genetic algorithm, Journal of Petroleum Science and Engineering77(3) (2011), 375–385.
29.
AminA.E., A novel classification model for cotton yarn quality based on trained neural network using genetic algorithm, Knowledge-Based Systems39(2) (2013), 124–132.
30.
SepehrS. and HesamA., Thermal modeling of gas engine driven air to water heat pump systems in heating mode using genetic algorithm and Artificial Neural Network methods, International Journal of Refrigeration36(8) (2013), 2262–2277.
31.
KangjiL. and HongyeS., Forecasting building energy consumption with hybrid genetic algorithm-hierarchical adaptive network-based fuzzy inference system, Energy and Buildiings42(11) (2010), 2070–2076.
32.
Hernández-LobatoJ.M. and RyanP.A., Probabilistic backpropagation for scalable learning of bayesian neural networks, in: 32nd International Conference on Machine Learning, Lille, France, JMLR: W&CP, Vol. 37, 2015.
33.
XueJ.Q., Optimization Theory and Method, Metallurgy Industry Press, Beijing, 2003.
34.
FredricM.H. and IvicaK., Principles of Neurocomputing for Science & Engineering, McGraw-Hill Companies, New York, 2003.
35.
FrankA. and AsuncionA., UCI machine learning repository [http://archive.ics.uci.edu/ml], School of Information and Computer Science, University of California, Irvine CA, 2010.
