Statistical Mechanics of Neural Computation

Abstract

Neural computation is a style of computation that draws inspiration from the way the brain computes. It is an in trinsically collective paradigm characterized by high con nectivity among a very large number of simple pro cessors running in parallel, possibly asynchronously. Methods developed in the theory of many-particle systems can be brought to bear on important conceptual questions about the operation and programming of such computational assemblies. This paper reviews several basic problems that arise in this area: the mathematical formulation of the collective computation done by such a network and of algorithms for programming ("teaching") them. The importance of phase transitions for understanding the generic behavior of such systems and algorithms is emphasized.

Get full access to this article

View all access options for this article.

References

Amit, D.J. , Gutfreund, H. , and Sompolinsky, H. 1985a. Phys. Rev. 32:1007-1018.

Amit, D.J. , Gutfreund, H. , and Sompolinsky, H. 1985b. Phys. Rev. Lett. 55:1530-1533.

Amit, D.J. , Gutfreund, H. , and Sompolinsky, H. 1987. Ann. Phys. 173:30.

Anderson, J.A. 1968. Kybernetik 5:113-119.

Anderson, J.A. 1970. Math. Biosci. 8:137-160.

Baum, E.B. , and Wilczek, F. 1988. Proc. IEEE Conf. on Neural Information Processing Systems, Denver, 1987.

Bohr, H. et al. 1988. Niels Bohr Inst. Report HE-88-25.

Cortes, C. , Krogh, A. , and Hertz, J.A. 1987. J. Phys. A 20:4449.

Denker, J. , et al. 1987. Complex Systems 1:877-922.

10.

Derrida, B. , Gardner, E. , and Zippelius, A. 1987. Europhys. Lett. 4:167-173.

11.

Gardner, E. 1987. Europhys. Lett. 4:481-485.

12.

Gardner, E. 1988. J. Phys. A 21:257-270.

13.

Gardner, E. , and Derrida, B. 1988. J. Phys. A 21:271-284.

14.

Glauber, R.G. 1963. J. Math. Phys. 4:294.

15.

Gorman, P. , and Sejnowski, T.J. 1988 Neural Networks 1:75-90.

16.

Hebb, D.O. 1949. The organization of behavior. New York : Wiley.

17.

Hertz, J.A. , Grinstein, G. , and Solla, S.A. 1987. In Heidelberg colloquium on glassy dynamics, edited by J. L. van Hemmen and I. Morgenstern. Berlin: Springer-Verlag , pp. 538-546.

18.

Hertz, J.A. , Thorbergsson, G.I. , and Krogh, A. 1988. Physica Scripta, in press.

19.

Hopfield, J.J. 1982. Proc. Natl. Acad. Sci. USA 79:2554-2558.

20.

Hopfield, J.J. 1984. Proc. Natl. Acad. Sci. USA 81:3088-3092.

21.

Hopfield, J.J. 1987. Proc. Natl. Acad. Sci. USA 84:8429-8433.

22.

Hopfield, J.J. , and Tank, D. 1985. Biol. Cybernetics 52:141-152.

23.

Kinzel, W. 1988. Physica Scripta, in press.

24.

Kohonen, T. 1974. IEEE Trans. C23:444-445.

25.

Kohonen, T. 1977. Associative memory: a system theoretical approach . Berlin: Springer-Verlag.

26.

Kohonen, T. 1984. Self organization and associative memory. Berlin: Springer-Verlag.

27.

Krogh, A. , and Hertz, J.A. 1988. J. Phys. A 21:2211-2224.

28.

Le Cun, Y. 1987. Ph.D. dissertation, University of Paris.

29.

Lehky, S. , and Sejnowski, T.J. 1988. Nature, in press. Little, W.A. 1974. Math. Biosci. 19:101-120.

30.

McCullough, W.S. , and Pitts, W. 1943. Bull. Math. Biophys. 5:115-133.

31.

Minsky, M. , and Papert, S. 1969. Perceptrons. Cambridge: MIT Press .

32.

Oja, E. 1980. Int. J. Systems Sci. 11:629-640.

33.

Parker, D.B. 1985. MIT Tech. Rept. TR-47. Cambridge : MIT Center for Computational Research in Economics and Management Science .

34.

Personnaz, L. , Guyon, I. , and Dreyfus, G. 1985. J. Phys. Lett. 46:L359.

35.

Qian, N. , and Sejnowski, T. 1988. J. Mol. Biol. 202:865-884.

36.

Rosenblatt, F. 1962. Principles of neurodynamics. New York : Spartan.

37.

Rumelhart, D.E. , Hinton, G.E. , and Williams, R.J. 1986. Nature 323:533-536.

38.

Sejnowski, T.J. , and Rosenberg, C.R. 1987 Complex Systems 1:145-168.

39.

Sompolinsky, H. 1987. In Heidelberg colloquium on glassy dynamics, edited by J. L. van Hemmen and I. Morgenstern. Berlin: Springer-Verlag , pp. 485-527.