A Model of Primate Visual-Motor Conditional Learning

Abstract

Observations of behavior and neural activity in premotor cortex of monkeys learning to pair an arbitrary visual stimulus with one of a set of previously learned behaviors are modeled with a network comprising a large number of motor selection columns. Reinforcement learning is used to recognize new visual patterns and acquire the appropriate visual-motor conditions. The architecture employs a distributed representation in which a single pattern is coded by a small subset of columns. A column is initially able to respond to many different inputs; as it learns to trigger a motor program, its responses become more narrowly defined. Each column's output is a set of votes for the various motor programs. The votes for each program are collected by selection units, which drive a winner-take-all circuit to determine whether a particular motor program is executed. The model is successful in reproducing the sequence of behavioral responses given by the subjects, as well as a number of phenomena that have been observed at the single-unit level. Finally, we offer a comparison to the backpropagation learning algorithm that demonstrates key principles which have been designed into our algorithm.

Keywords

conditional learning reinforcement learning cortical columns,distributed representations premotor cortex primates

Get full access to this article

View all access options for this article.

References

Amari, S. , & Arbib, M.A. (1977). Competition and cooperation in neural nets. In J. Metzler (Ed.), Systems neuroscience . New York: Academic Press.

Arbib, M.A. (1989). The metaphorical brain 2: Neural networks and beyond. New York: John Wiley & Sons.

Arbib, M.A. (1990). Programs, schemas, and neural networks for control of hand movements: Beyond the RS framework. In M. Jeannerod (Ed.), Attention and performance XIII. Motor representation and control. Hillsdale, NJ: Lawrence Erlbaum Associates.

Barto, A.G. , Sutton, R.S. , & Anderson, C.W. (1983). Neuron-like adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 834-846.

Bush, R.R. , & Mosteller, F. (1958). Stochastic models for learning. New York: John Wiley and Sons.

Didday, R.L. (1976). A model of visuomotor mechanisms in the frog optic tectum. Mathematical Biosciences, 30, 169-180.

Grossberg, S. (1976). A theory of visual coding, memory, and development: Part 1. Parallel development and coding of neural feature detectors. Biological Cybernetics, 23, 121-134.

Hinton, G.E. , & Sejnowski, T.J. (1986). Learning and relearning in Boltzmann machines. In J. L. McClelland & D. E. Rumelhart (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Vol. 1. Foundations. Cambridge, MA: Bradford Books/The MIT Press.

Mitz, A.R. , Godshalk, M. , and Wise, S.P. (1991). Learning-dependent neuronal activity in the premotor cortex. Activity during the acquisition of conditional motor associations . Journal of Neuroscience, 11 (6), 1855-1872.

10.

Rizzolatti, G. (1987). Functional organization of inferior area 6. In Motor Areas of the Cerebral Cortex (CIBA Foundation Symposium 132). Chichester: John Wiley and Sons, 171-186.

11.

Rumelhart, D.E. , Hinton, G.E. , & Williams, R.J. (1986). Learning internal representations by error propagation. In J. L. McClelland & D. E. Rumelhart (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Vol. 1. Foundations. Cambridge, MA: Bradford Books/The MIT Press, 318-362.

12.

Rumelhart, D.E. , & Zipser, D. (1986). Feature discovery by competitive learning. In J. L. McClelland & D. E. Rumelhart (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, Vol. 1. Foundations. Cambridge, MA: Bradford Book/ The MIT Press.

13.

Taira, M. , Mine, S. , Georgopoulos, A.P. , Murata, A. , & Sakata, H. (1990). Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Experimental Brain Research, 83, 29-36.

14.

von der Malsburg, C. (1973). Self-organizing of orientation sensitive cells in the striate cortex. Kybernetik, 14, 85-100.

15.

Weitzenfeld, A. (1991). NSL: Neural simulation language, version 2.1, Technical Report 91-05. Los Angeles: Center for Neural Engineering, University of Southern California.

16.

Williams, R.J. (1988). Toward a theory of reinforcement learning connectionist systems, Technical Report NU-CC5-88-3.