We would like the behavior of the artificial agents that we construct to be as well-adapted to their environments as natural animals are to theirs. Unfortunately, designing controllers with these properties is a very difficult task. In this article, we demonstrate that continuous-time recurrent neural networks are a viable mechanism for adaptive agent control and that the genetic algorithm can be used to evolve effective neural controllers. A significant advantage of this approach is that one need specify only a measure of an agent's overall performance rather than the precise motor output trajectories by which it is achieved. By manipulating the performance evaluation, one can place selective pressure on the development of controllers with desired properties. Several novel controllers have been evolved, including a chemotaxis controller that switches between different strategies depending on environmental conditions, and a locomotion controller that takes advantage of sensory feedback if available but that can operate in its absence if necessary.
Agre, P., & Chapman, D. (1987). Pengi: an implementation of a theory of activity . Proceedings of the Sixth National Conference on Artificial Intelligence, pp. 268-272. San Mateo, CA: Morgan Kaufmann.
2.
Anderson, J. A., & Rosenfeld, E. (Eds.). (1988). Neurocomputing: Foundations of research. Cambridge, MA: The MIT Press.
3.
Baker, J.E. (1987). Reducing bias and inefficiency in the selection algorithm. In J. J. Grefenstette (Ed.), Proceedings of the Second International Conference on Genetic Algorithms (pp. 14-21). Hillsdale, NJ: Lawrence Erlbaum Associates.
4.
Beer, R.D. (1990). Intelligence as adaptive behavior: An experiment in computational neuroethology. New York: Academic Press.
5.
Beer, R.D., Chiel, H.J., & Sterling, L.S. (1989). Heterogeneous neural networks for adaptive behavior in dynamic environments. In D. S. Touretzky (Ed.), Advances in neural information processing systems, Vol. 1. San Mateo, CA: Morgan Kaufmann.
6.
Belew, R.K., McInerney, J., & Schraudolph, N.N. (1991). Evolving networks: Using the genetic algorithm with connectionist learning. In C. G. Langton, C. Taylor, J. D. Farmer, & S. Rasmussen (Eds.), Artificial life II. Reading, MA: Addison-Wesley.
7.
Booker, L.B. (1988). Classifier systems that learn internal world models . Machine Learning, 3, 161-192.
8.
Brooks, R.A. (1986). A robust layered control system for a mobile robot . IEEE Journal of Robotics and Automation, RA-2, 14-23.
9.
Brooks, R.A. (1989). A robot that walks: Emergent behaviors from a carefully evolved network. Neural Computation, 1, 253-262.
10.
Brooks, R.A. (1991). Intelligence without representation. Artificial Intelligence, 47, 139-159.
11.
Caruana, R., & Schaffer, J.D. (1988). Representation and hidden bias: Gray vs. binary coding for genetic algorithms. In Proceedings of the Fifth International Conference on Machine Learning. San Mateo, CA: Morgan Kaufmann.
12.
Chiel, H.J., & Beer, R.D. (1989). A lesion study of a heterogeneous neural network for hexapod locomotion. In Proceedings of the International Joint Conference on Neural Networks (pp. 407-414). IEEE.
13.
Collins, R.J., & Jefferson, D.R. (1991). Representations for artificial organisms. In J. A. Meyer and S. W Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press.
14.
de Garis, H. (1990). Genetic programming: Building artificial nervous systems using genetically programmed neural network modules. In B. W Porter & R. J. Mooney (Eds.), Proceedings of the Seventh International Conference on Machine Learning (pp. 132-139). San Mateo, CA : Morgan Kaufmann.
15.
Delcomyn, F. (1980). Neural basis of rhythmic behavior in animals. Science, 210, 492-498.
16.
Fogel, D.B., & Atmar, J.W. (1990). Comparing genetic operators with gaussian mutations in simulated evolutionary processes using linear systems. Biological Cybernetics, 63, 111-114.
17.
Getting, P.A., & Dekin, M.S. (1985). Tritonia swimming: A model system for integration within rhythmic motor systems. In A. I. Selverston (Ed.), Model neural networks and behavior. New York: Plenum Press.
18.
Goldberg, D.E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley.
19.
Graham, D. (1985). Pattern and control of walking in insects. Advances in Insect Physiology, 18, 31-140.
20.
Harp, S.A., Samad, T., & Guha, A. (1989). Towards the genetic synthesis of neural networks . In J. D. Schaffer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms (pp. 360-369). San Mateo, CA: Morgan Kaufmann.
21.
Holland, J. (1975). Adaptation in natural and artificial systems. Ann Arbor: University of Michigan Press.
22.
Hopfield, J.J. (1984). Neurons with graded response properties have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences of the United States of America , 81, 3088-3092.
23.
Kaelbling, L., & Rosenschein, S. (1990). Action and planning in embedded agents. Journal of Robotics and Autonomous Systems , 6, 35-48.
24.
Langton, C. G., Taylor, C. , Farmer, J. D., & Rasmussen, S. (Eds.). (1991). Artificial Life II. Reading, MA: Addison-Wesley.
25.
Maes, P. (1989). The dynamics of action selection. Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (pp. 991-997). San Mateo, CA: Morgan Kaufmann.
26.
Maes, P., & Brooks, R.A. (1990). Learning to coordinate behaviors. Proceedings of the Eighth National Conference on Artificial Intelligence [AAAI-90] (pp. 796-802). San Mateo, CA: Morgan Kaufmann.
27.
Maes, P. (Ed.). (1991). Designing autonomous agents. Cambridge, MA: The MIT Press.
28.
McFarland, D. (1981). The Oxford companion to animal behavior. Oxford, Engl.: Oxford University Press.
29.
Meyer, J. A., & Wilson, S. W (Eds.). (1991). From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press.
30.
Miller, G.F., Todd, P.M., & Hegde, S.U. (1989). Designing neural networks using genetic algorithms . In J. D. Schaffer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms (pp. 379-384). San Mateo, CA: Morgan Kaufmann.
31.
Packard, N.H. (1988). Intrinsic adaptation in a simple model for evolution . In C. Langton (Ed.), Artificial life. Reading, MA: Addison-Wesley .
32.
Pearlmutter, B.A. (1989). Learning state space trajectories in recurrent neural networks. Neural Computation, 1, 263-269.
33.
Pearlmutter, B.A. (1990). Dynamic recurrent neural networks (Technical Report CMU-CS-90-196). Pittsburgh: School of Computer Science, Carnegie Mellon University.
34.
Pearson, K.G. (1976). The control of walking. Scientific American, 235, 72-86.
35.
Pearson, K.G., Fourtner, C.R., & Wong, R.K. (1973). Nervous control of walking in the cockroach. In R. B. Stein, K. G. Pearson, R. S. Smith, & J. B. Redford (Eds.), Control of posture and locomotion. New York: Plenum Press.
36.
Rizki, M.M., & Conrad, M. (1986). Computing the theory of evolution. Physica D, 22, 83-99.
37.
Schaffer, J.D., Caruana, R.A., Eschelman, L.J., & Das, R. (1989). A study of control parameters affecting online performance of genetic algorithms for function optimization. In J. D. Schaffer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms (pp. 51-60). San Mateo, CA: Morgan Kaufmann.
38.
Schraudolph, N.N., & Belew, R.K. (1990). Dynamic parameter encoding for genetic algorithms (Technical Report CS 90-175). San Diego: Cognitive Computer Science Research Group, Computer Science and Engineering Dept., University of California at San Diego.
39.
Selverston, A. I., & Moulins, M. (Eds.). (1987). The crustacean stomatogastric system. Berlin: Springer-Verlag.
40.
Travers, M. (1988). Animal construction kits. In C. Langton (Ed.), Artificial life. Reading, MA: Addison-Wesley.
41.
Werner, G.M., & Dyer, M.G. (1991). Evolution of communication in artificial organisms . In C. G. Langton, C. Taylor, J. D. Farmer, & S. Rasmussen (Eds.), Artificial life II. Reading, MA: Addison-Wesley.
42.
Whitley, D., & Hanson, T. (1989). Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer (Ed.), Proceedings of the Third International Conference on Genetic Algorithms (pp. 391-396). San Mateo, CA: Morgan Kaufmann.
43.
Wieland, A.P. (1990). Evolving controls for unstable systems. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski, & G. E. Hinton (Eds.), Connectionist models: Proceedings of the 1990 Summer School (pp. 91-102). San Mateo, CA: Morgan Kaufinann.
44.
Wilson, S.W. (1986). Knowledge growth in an artificial animal. In K. S. Narendra (Ed.), Adaptive and learning systems. New York: Plenum Press.
45.
Winograd, T., & Flores, F. (1986). Understanding computers and cognition. Norwood, NJ: Ablex.
