A new weight initialization method for sigmoidal FFANN

Abstract

Weight initialization is the most important component which affects the performance of artificial neural network during training the network using Back-propagation algorithm. The initial starting weights have significant effect on the training. If the weights are too large then the sigmoid will saturate, that makes learning slow. If weights are too small then gradients are also too small. In this paper a new weight initialization method has been proposed. The results for the proposed weight initialization technique are compared against the random weight initialization method. In this paper the proposed weight initialization method is statistically analyzed. Ten different data sets out of which five sets of data are taken from UCI machine learning repository and five sets of data are generated using function approximation problems that are used. Resilient Back Propagation training algorithm is used for training the feed forward artificial neural network. The proposed weight initialization method gives better results when compared with random weight initialization technique.