Parallel Algorithms for the Training Process of a Neural Network-Based System

Abstract

This paper addresses the problem of developing efficient parallel algorithms for the training procedure of a neural network-based Fingerprint Image Comparison (FIC) system. The target architecture is assumed to be a coarse-grain distributed-memory parallel architecture. Two types of parallelism—node parallelism and training set parallelism (TSP)—are investigated. Theoretical analysis and experimental results show that node parallelism has low speedup and poor scalability, while TSP proves to have the best speedup performance. TSP, however, is amenable to a slow convergence rate. To reduce this effect, a modified training set parallel algorithm using weighted contributions of synaptic connections is proposed. Experimental results show that this algorithm provides a fast convergence rate while keeping the best speedup performance obtained. The combination of TSP with node parallelism is also investigated. A good performance is achieved by this approach. This provides better scalability with the trade-off of a slight decrease in speedup. The above algorithms are implemented on a 32-node CM-5.

Get full access to this article

View all access options for this article.

References

Baldi, P. , and Chauvin, Y. 1993. Neural networks for fingerprint recognition . Neural Computation 5: 402-418 .

Ballard and Brown . 1982. Computer Vision. Englewood Cliffs, NJ: Prentice Hall .

Haykin, S. 1994. Neural Networks: A Comprehensive Foundation. New York: Macmillan .

Pomerleau, D. A. , Gusciora, G. L. , Touretzky, D. S. , and Kung, H. T. 1988. Neural network simulation at warp speed: How we got 17 million connections per second . Proceedings of the IEEE International Conference on Neural Networks, pp. 143-150 .

Rosenberg, C. R. , and Blelloch, G. 1987. An implementation of network learning on the Connection Machine . Proceedings of 10th International Conference on Artificial Intelligence, pp. 329-340 .

Serbedzija, N. B. 1996. Simulating artificial neural networks on parallel architectures. IEEE Computer Special Issue on Neural Computing, March, pp. 56-63.

Singer, A. 1990. Implementations of artificial neural networks on the Connection Machine . Parallel Computing 14: 305-315 .

Tørresen, J. , Mori, S. , Nakashima, H. , Tomita, S. , and Landsverk, O. 1995. Exploiting multiple degrees of BP parallelism on the highly parallel computer AP1000 . Paper presented at the Fourth International Conference on Artificial Neural Networks.

Wegstein, Rafferty , and Pencak . 1968. Matching fingerprints by computer. Technical Note 466, National Bureau of Standards .

10.

Wilson, C. L. , Candela, G. , Grother, P. J. , Watson, C. I. , and Wilkinson, R. A. 1992. Massively parallel neural network fingerprint classification system. National Institute of Standards and Technology , July.

11.

Zeng, S. 1994. The application of parallel processing techniques to neural network based fingerprint recognition systems. Ph.D. thesis, Department of Electrical and Computer Engineering, West Virginia University.

12.

Zhang, X. , Mckenna, M. , Mesirov, J. F. , and Waltz, D. 1990. An efficient implementation of the backpropogation algorithm on the Connection Machine CM-2 . Parallel Computing 14: 317-327 .