14.arch/JAF.gomes
.m From gomes@ICSI.Berkeley.EDU Sat Sep 18 14:46:14 1993
.ls 2
.na
.LP
Mapping Neural Networks onto Parallel Hardware

Benedict A. Gomes 

(Professor J. A. Feldman)

Recent years have seen an explosion in the application of
neural network technology to a variety of applications, ranging from
speech and vision to cognitive modeling. The computational cost of
neural network learning algorithms and the inherent parallelism
motivate the use of parallel hardware. Neural networks may be viewed
as arbitrarily structured computation graphs with many levels and
types of parallelism. The optimal mapping\(emassignment and
scheduling\(emof such graphs onto the communication network of a given
parallel processor is known to be NP-hard. Heuristic optimization
methods such as branch and bound, A*, simulated annealing [1], and even
neural nets have been applied to this problem.

We intend to apply these general mapping methods, and develop
other methods as the application demands, to the problem of mapping
neural networks (dense and sparse) onto highly parallel (100 to 1000
processors) hardware such as the CM-5 and the Connectionist Neural
Supercomputer (CNS-1) [2] under development here. These mapping
algorithms will be implemented in a high-level connectionist simulator
written in the new object-oriented language, pSather [3]. The simulator
will be based on the serial simulator ICSIM [4] written in Sather.

[1]	F. Ercal, J. Ramanujam, and P. Sadayappan, "Task Allocation onto a
Hypercube by Recursive Mincut Bipartitioning," 3rd Conf. 
Hypercube Concurrent Computers and Applications, Pasadena, CA,
1988.

[2]	K. Asanovic, J. Beck, T. Callahan, J. Feldman, B. Irissou, B. Kingsbury, P. Kohn, J. Lazzaro, N. Morgan, D. Stoutamire, and J. Wawrzynek, CNS-1 Architecture Specification, Int. Comp.
Sci. Inst., Report No. TR-93-021.

[3]	S. Murer, J. A. Feldman, and C. Lim, pSather:  Layered Extensions to
an Object-Oriented Language for Efficient Parallel
Computation, Int. Comp. Sci. Inst., Report
No. TR-93-028.

[4]	H. W. Schmidt and B. Gomes, ICSIM: An Object-Oriented
Connectionist Simulator, Int. Comp. Sci. Inst.,
Report No. TR-91-048.