Automatic Definition of Modular Neural Networks

Abstract

This article illustrates an artificial developmental system that is a computationally efficient technique for the automatic generation of complex artificial neural networks (ANNs). The artificial developmental system can develop a graph grammar into a modular ANN made of a combination of simpler subnetworks. A genetic algorithm is used to evolve coded grammars that generate ANNs for controlling six-legged robot locomotion. A mechanism for the automatic definition of neural subnetworks is incorporated Using this mechanism, the genetic algorithm can automatically decompose a problem into subproblems, generate a subANN for solving the subproblem, and instantiate copies of this subANN to build a higher-level ANN that solves the problem. We report some simulation results showing that the same problem cannot be solved if the mechanism for automatic definition of subnetworks is suppressed. We support our argument with pictures that describe the steps of development, how ANN structures are evolved, and how the ANNs compute.

Keywords

animats cellular encoding modularity locomotion automatic definition of neural subnetworks

Get full access to this article

View all access options for this article.

References

Ackley, D. , & Littman, M. (1991). The interaction between learning and evolution . In C. G. Langton , C. Taylor , J. D. Farmer , & S. Rasmussen (Eds.), Artificial Life II. Reading, MA: Addison-Wesley.

Beer, R. (1990). Intelligence as adaptive behavior. New York: Academic Press.

Beer, R. , & Gallagher, J. (1992). Evolving dynamical neural networks for adaptive behavior. Adaptive Behavior, 1, 92-122.

Belew, R. (1993). Interposing an ontogenic model between genetic algorithms and neural networks. In R. Belew , Neural information processing systems (5th ed.).

Boers, E. J. W , & Kuiper, H. (1992). Biological metaphor and the design of modular artificial neural networks. Unpublished master's thesis, Leiden

University, the Netherlands.

Brooks, R. (1991). Intelligence without representation. Artificial Intelligence, 47, 139-159.

Cliff, D. , Harvey, I. , & Husbands, P. (1993). Explorations in evolutionary robotics. Adaptive Behavior, 2, 73-110.

Collins, R. , & Jefferson, D. (1991). Selection in massively parallel genetic algorithm . In R. Belew (Ed.), Proceedings of the Fourth International Conference on Genetic Algorithms

Dellaert, F. , & Beer, D. (1994). Co-evolving body and brain in autonomous agent using a developmental model (Ces94-16). Cleveland: Case Western University.

10.

Gruau, F. (1992). Genetic synthesis of Boolean neural networks with a cell rewriting developmental process. In Combination of genetic algorithms and neural networks.

11.

Gruau, F. (1993). The mixed parallel genetic algorithm. Parallel Computing, 93.

12.

Gruau, F. (1994). Neural network synthesis using cellular encoding and the genetic algorithm. Doctoral thesis, Ecole Normale Supérieure de Lyon.

13.

Gruau, F. (1995). Genetic programming of Neural Network: Theory and Practice. In Intelligent Hybrid Systems, S. Goonatilake , ed. J. Wiley.

14.

Gruau, F. , Ratajszczak, J. , & Wiber, G. (1994). A neural compiler. Theoretical Computer Science, pp. 2-52.

15.

Gruau, F. , & Whitley, D. (1993). Adding learning to the cellular developmental process: A comparative study. Evolutionary Computation, 1(3), 213-234.

16.

Hinton, G.E. , & Nowlan, S.J. (1987). How learning can guide evolution. Complex Systems, 4, 495-502.

17.

Hubel, D. , and Wiesel, T. (1979). Brain mechanisms of vision. Scientific American.

18.

International Conference on Graph Grammars [lecture note]. (1990). Computer Science, 532.

19.

Kitano, H. (1992). Designing neural network using genetic algorithm with graph generation system. Complex Systems, 4, 461-476.

20.

Kodjabachian, J. , & Meyer, J. (1994). Development, learning and evolution in animats . In PerAc '94. IEEE Computer Society Press.

21.

Koza, J.R. (1992). Genetic programming: A paradigm for genetically breeding a computer population of computer programs to solve problems. Cambridge: MIT Press.

22.

Koza, J.R. (1994). Genetic programming II: Automatic discovery of reusable programs. Cambridge: MIT Press.

23.

Meyer, J.A. , & Guillot, A. (1994). From SAB90 to SAB94: Four Years of Animat Research. In Proceedings of the Third International Conference on Simulation of Adaptive Behavior Cambridge, MA: MIT Press .

24.

Mjolness, E. , Sharp, D. , & Alpert, B. (1988). Scaling, machine learning and genetic neural nets (Rep. No. La-ur-88-142). Los Alamos, NM: Los Alamos National Laboratory.

25.

Muehlenbein, H. , Schomish, M. , & Born, J. (1991). The parallel genetic algorithm as function optimizer . Parallel Computing, 17, 619-632.

26.

Muller-Wilm, U. , Cruse, H. , & Dean, J. (1993). Artificial neural nets for controlling a 6-legged walking system. In Proceedings of the Second International Conference on Simulation of Adaptive Behavior Cambridge, MA: MIT Press.

27.

Parisi, D. , & Nolfi, S. (1992). Morphogenesis of neural networks. (Tech. Rep.) . Rome: University of Rome.

28.

Prusinkiewicz, P. , & Lindenmayer, A. (1992). The logarithmic beauty of plants. Springer-Verlag.

29.

Sims, K. (1994). Evolving 3d morphology and behavior by competition . In R. Brook and P Maes (Eds.), Proceedings of the Fourth International Conference on Artificial Life. Cambridge: MIT Press.

30.

Vaario, J. (1993). An emergent modeling method for artificial neural networks. Doctoral thesis, University of Tokyo.