Optimal and Suboptimal Pattern Allocation Schemes for Backpropagation Learning on a Heterogeneous Processor Network

Abstract

This article investigates optimal and suboptimal pattern allocation schemes when training-set parallelism is used as the paradigm to map a backpropagation neural network on a heterogeneous array of processors. In earlier work, it was shown that finding the optimal allocation of patterns to minimize the time for a training epoch for such a mapping leads to a mixed integer programming problem. Because the solution to the mixed integer programming requires prohibitively large computing time, several suboptimal allocation methods that are computationally less demanding are discussed and their performance compared with the optimal solution for the NETTALK benchmark problem. The processor network used for mapping the neural network is a heterogeneous array of transputers connected in a pipelined ring topology.