In this article, we evolve and analyze continuous-time recurrent neural networks capable of associating the smells of different foods with edibility or inedibility in different environments. First, we present an in-depth analysis of this task, highlighting the evolutionary challenges it poses and how these challenges informed our experimental design. Next, we describe the evolution of nonplastic neural circuits that can solve this food edibility learning problem. We then show that the dynamics of the best evolved nonplastic circuits instantiate finite state machines that capture the combinatorial structure of this task. Finally, we demonstrate that successful circuits with Hebbian synaptic plasticity can also be evolved, but that such circuits do not utilize their synaptic plasticity in a traditional way.
Baxter, D.A., & Byrne, J.H. (1993). Learning rules from neurobiology. In D. Gardner (Ed.), The neurobiology of neural networks (pp. 71—105). Cambridge, MA: MIT Press.
3.
Beer, R.D. (1997). The dynamics of adaptive behavior: A research program. Robotics and AutonomousSystems20: 257—289.
Bi, G-Q., & Rubin, J. (2005). Timing in synaptic plasticity: From detection to integration. Trends in Neurosciences28: 222—228.
6.
Blynel, J., & Floreano, D. (2003). Exploring the T-maze: Evolving learning-like robot behaviors using CTRNNs. In C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, & E. Costa (Eds.), Applications of evolutionary computing (Lecture Notes in Computer Science 2611) (pp. 593—604). Berlin: Springer.
7.
Boal, J.G., Dunham, A.W., Williams, K.T., & Hanlon, R.T. (2000). Experimental evidence for spatial learning in Octopuses (Octopus bimaculoides). J. Comp. Psych.114: 246—252.
8.
Bullock, T.H., & Horridge, G.A. (1965). Structure and function in the nervous systems of invertebrates. W.H. Freeman.
9.
Casey, M. (1996). The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction . Neural Computation8: 1135— 1178.
10.
Chalmers, D.J. (1991). The evolution of learning: An experiment in genetic connectionism. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski & G. E. Hinton (Eds.), Connectionist models: Proceedings of the 1990 Summer School (pp. 81—90). San Mateo: Morgan Kaufmann.
11.
Chiel, H.J., & Susswein, A.J. (1993). Learning that food is inedible in freely-behaving Aplysia californica, Behavioral Neuroscience107: 327—338.
12.
Cotter, N.E., & Conwell, P.R. (1990). Fixed-weight networks can learn. In Proceedings of the International Joint. Conference on Neural Networks (pp. 11553—11559). IEEE Press.
13.
Dan, Y., & Poo, M-M. (2004). Spike timing-dependent plasticity of neural circuits . Neuron44: 23—30.
14.
Dan, Y., & Poo, M-M. (2006). Spike timing-dependent plasticity: From synapse to perception. Physiological Review86: 1033—1048.
15.
Destexhe, A., & Marder, E. (2004). Plasticity in single neuron and circuit computations . Nature431: 789—795.
16.
Feldkamp, L.A., Puskorius, G.V., & Moore, P.C. (1996). Adaptation from fixed weight dynamic networks . In Proceedings of the IEEE International Conference on Neural Networks (pp. 155—160). IEEE Press.
17.
Floreano, D., & Mondada, F. (1996). Evolution of plastic neurocontrollers for situated agents. In P. Maes, M. Mataric, J. Meyer, J. Pollack, & S. Wilson (Eds.), From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior (pp. 402— 410). Cambridge, MA: MIT Press.
18.
Floreano, D., & Urzelai, J. (2001). Evolution of plastic control networks. Autonomous Robotics11: 311—317.
19.
Funahashi, K.I., & Nakamura, Y. (1993). Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks6: 801—806.
20.
Izquierdo-Torres, E., & Harvey, I. (2006). Learning on a continuum in evolved dynamical node networks. In L. Rocha et al. (Eds.), Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems (pp. 507—512). Cambridge, MA: MIT Press.
21.
Kandel, E.R., Kupfermann, I., & Iverson, S. (2000). Learning and memory. In E. R. Kandel, J. H. Schwartz, & T. M. Jessell (Eds.), Principles of neural science, 4th ed. (pp. 1227—1246). McGraw-Hill.
22.
Kandel, E.R., & Siegelbaum, S.A. (2000). Transmitter release. In E. R. Kandel, J. H. Schwartz, & T. M. Jessell (Eds.), Principles of neural science, 4th ed. (pp. 253—279). McGraw-Hill.
23.
Kelly, A.M.C., & Garavan, H. (2005). Human functional neuroimaging of brain changes associated with practice. Cerebral Cortex15: 1089—1102.
24.
Kimura, M., & Nakano, R. (1998). Learning dynamical systems by recurrent neural networks from orbits. Neural Networks11: 1589—1599.
25.
Llinas, R.R. (1988). The intrinsic electrophysiological properties of mammalian neurons: Insights into central nervous system function. Science. 242: 1654—1664.
26.
Lockery, S.R., & Sejnowski, T.J. (1993). Voyages through weight space: Network models of an escape reflex in the leech. In R. D. Beer, R. E. Ritzmann & T. McKenna (Eds.), Biological neural networks in invertebrate neuroethology and robotics (pp. 251—266). San Diego: Academic Press.
27.
Major, G., & Tank, D. (2004). Persistent neural activity: prevalence and mechanisms. Current Opinion in Neurobiology14: 675—684.
28.
Marder, E., Abbott, L.F., Turrigiano, G.G., Liu, Z., & Golowasch, J. (1996). Memory from the dynamics of intrinsic membrane currents. Proc. Nat. Acad. Sci. USA93: 13481— 13486.
29.
Margulies, C., Tully, T., & Dubnau, J. (2005). Deconstructing memory in Drosophila. Current Biology15: R700—R713.
30.
McFarland, D. (Ed.) (1981). The Oxford companion to animal behavior. Oxford University Press.
31.
McIntosh, A.R. (1999). Mapping cognition to the brain through neural interactions. Memory7: 523—548.
32.
Miller, G.F., & Todd, P.M. (1991). Exploring adaptive agency: I. Theory and methods for simulating the evolution of learning. In D. S. Touretzky, J. L. Elman, T. J. Sejnowski & G. E. Hinton (Eds.), Connectionist models: Proceedings of the 1990 summer school (pp. 65—80). San Mateo: Morgan Kaufmann.
33.
Münte, T.F., Altenmüller, E., & Jäncke, L. (2002). The musician's brain as a model of neuroplasticity . Nature Reviews Neuroscience3: 473—478.
34.
Phattanasri, P. (2002). Associative learning in evolved dynamical neural networks. Ph.D. Dissertation, Department of Electrical Engineering and Computer Science, Case Western Reserve University.
35.
Susswein, A.J., Schwarz, M., & Feldman, E. (1986). Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. Journal of Neuroscience6: 1513— 1527
36.
Sutton, M.A., & Carew, T.J. (2002). Behavioral, cellular and molecular analysis of memory in Aplysia I: Intermediate-term memory. Integrative and ComparativeBiology42: 725—735.
37.
Thorndike, E.L. (1898). Animal intelligence: An experimental study of the associative processes in animals. The Psychological Review, Series of Monographs Supplements (Supplement 2) 4: 1—109.
38.
Todd, P.M., & Miller, G.F. (1991a). Exploring adaptive agency II: Simulating the evolution of associative learning. In J.A. Meyer & S.A. Wilson (Eds.), From Animals to Animats 1: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 306—315). Cambridge, MA: MIT Press.
39.
Todd, P.M., & Miller, G.F. (1991b). Exploring adaptive agency III: Simulating the evolution of habituation and sensitization. In H. P. Schwefel & R. Manner (Eds.), Proceedings of the First International Conference on Parallel Problem Solving From Nature (pp. 307—313). Berlin : Springer.
40.
Toledo-Rodriguez, M., El Manira, A., Wallen, P., Svirskis, G., & Hounsgaard, J. (2005). Cellular signaling properties in microcircuits. Trends in Neurosciences28: 534—540.
41.
Trewavas, A. (2003). Aspects of plant intelligence. Annals of Botany92: 1—20.
42.
Tuci, E., Quinn, M., & Harvey, I. (2002). An evolutionary ecological approach to the study of learning behavior using a robot-based model. Adaptive Behavior10: 201—221.
43.
Wood, D.C. (1969). Parametric studies of the response decrement produced by mechanical stimuli in the protozoan Stentor coeruleus. J. Neurobiology1: 345—360.
44.
Yamauchi, B., & Beer, R.D. (1994a). Sequential behavior and learning in evolved dynamical neural networks. Adaptive Behavior2: 219—246.
45.
Yamauchi, B., & Beer, R.D. (1994b). Integrating reactive, sequential and learning behavior using dynamical neural networks. In D. Cliff, P. Husbands, J. Meyer & S. Wilson (Eds.), From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (pp. 382—391). Cambridge, MA: MIT Press.
Zečević, D.Wu, J-Y., Cohen, L.B., London, J.A., Höpp, H-P., & Falk, C.X. (1989). Hundreds of neurons in the Aplysia abdominal ganglion are active during the gill-withdrawal reflex. J. Neuroscience9: 3681—3689.
