A Concurrent Adaptive Conjugate Gradient Learning Algorithm On Mimd Shared-Memory Machines

Abstract

A concurrent adaptive conjugate gradient learning al gorithm has been developed for training of multilayer feed-forward neural networks and implemented in C on a MIMD shared-memory machine (CRAY Y-MP/8- 864 supercomputer). The learning algorithm has been applied to the domain of image recognition. The per formance of the algorithm has been evaluated by ap plying it to two large-scale training examples with 2,304 training instances. The concurrent adaptive neural networks algorithm has superior convergence property compared with the concurrent momentum back-propagation algorithm. A maximum speedup of about 7.9 is achieved using eight processors for a large network with 4,160 links as a result of microtask ing only. When vectorization is combined with micro tasking, a maximum speedup of about 44 is realized using eight processors.

Get full access to this article

View all access options for this article.

References

Cray Research. 1991. Cray standard C programmer's reference manual. Mendota Heights, Minn.: Cray Research, Inc.

Dennis, J.E. , Jr., and Schnabel, R.B. 1983. Numerical methods for unconstrained optimization and nonlinear equations. Englewood Cliffs, N.J.: Prentice-Hall.

Hsu, H.-L. , and Adeli, H. 1991. A microtasking algorithm for optimization of structures . Internat. J. Supercomput. Appl. 5(2): 81-90.

Hung, S.L. , and Adeli, H. 1991a. A model of perceptron learning with a hidden layer for engineering design. Neuro-computing 3(1):3-14.

Hung, S.L. , and Adeli, H. 1991b. A hybrid learning algorithm for distributed memory multicomputers . Heuristics—The Journal of Knowledge Engineering 4(4):58-68.

Nocedal, J. 1990. The performance of several algorithms for large scale unconstrained optimization. In Large-scale numerical optimization, edited by T. F. Coleman and Y. Li. Philadelphia: SIAM, pp. 138-151.

Powell, M.J.D. 1986. Convergence properties of algorithms for nonlinear optimization . SIAM Rev. 28(4):487-500.

Rosenberg, C.R. , and Blelloch, G. 1988. An implementation of network learning on the Connection Machine. In Connectionist models and their implications: readings from cognitive science, edited by D. Waltz and J. A. Feldman. Norwood, N.J.: Ablex Publishing Corp.

Rumelhart, D.E. , Hinton, G.E. , and Williams, R.J. 1986. Learning internal representation by error propagation . In Parallel distributed processing, edited by D. E. Rumelhart and J. L. McClelland. Cambridge, Mass. : MIT Press, pp.318-362.