We present a methodology for incremental development of complex adaptive behaviors in robots. The methodology decomposes a given root strategy into a tree of self-contained supporting strategies that can be fully implemented and tested before the next strategy is added. The methodology also uses comparative performance tests for each new strategy relative to its predecessor. The methodology assumes the use of skill modules that can be shared by multiple behavioral layers and produces learning mechanisms that are highly specialized and context dependent. Two example applications of the methodology are presented using simulated robots in the domains of foraging/mapping and conflict resolution. The examples are implemented by hand using a decomposed model of behavior that allows skill modules to be shared while retaining a unique representation of each strategy for excitation and inhibition. Finally, we discuss how the solutions produced using this methodology differ from existing behavior-based solutions.
Andre, D. , & Russel, S. J. (2001). Programmable reinforcement learning agents . In T. G. Dietterich , S. Becker , & Z. Ghahramani (Eds.). Proceedings of the 13th Conference on Neural Information Processing Systems (NIPS’01). Vancouver, Canada (pp. 1019-1025 ). Cambridge, MA: MIT Press.
2.
Arkin, R. C. (1989). Motor schema-based mobile robot navigation . International Journal of Robotics Research, 8(4), 92-112 .
3.
Asada, M. , Noda, S. , Tawaratsumida, S. , & Hosoda, K. (1995). Vision-based reinforcement learning for purposive behavior acquisitions . In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’95) (pp. 146-153 ). IEEE.
Breazeal, C. , & Scassellati, B. (1998). Infant-like social interactions between a robot and a human caregiver . Adaptive Behavior, 8(1), 49-74 .
6.
Brooks, R. A. (1986). A robust layered control system for a mobile robot . IEEE Journal of Robotics and Automation, 2, 14-23 .
7.
Bryson, J. J. (2001). Intelligence by design: Principles of modularity and coordination for engineering complex adaptive agents. Ph.D. thesis, Massachusetts Institute of Technology, Artificial Intelligence Laboratory, Cambridge, MA.
8.
Bryson, J. J. , & Stein, L. A. (2001). Modularity and design in reactive intelligence . In B. Nebel (Ed.), Proceedings of the 17th Intenational Joint Conference on Artificial Intelligence (IJCAI’01). Seattle, WA. (pp. 1115-1120 ). Morgan Kaufmann.
9.
Cockburn, A. (2001). Agile software development. Boston, MA: Addison-Wesley .
10.
Cohen, L. B. , Chaput, H. H. , & Cashon, C. H. (2002). A constructivist model of infant cognition . Cognitive Development, 17, 1323-1343 .
11.
Dahl, T. S. , & Giraud-Carrier, C. (2001). PLANCS: Classes for programming adaptive behaviour based robots . In Proceedings of the 2001 Convention on Artificial Intelligence and the Study of Simulated Behaviour (AISB’01), Symposium on Nonconscious Intelligence: From natural to artificial (pp. 9-20 ). AISB.
12.
Enquist, M. (1985). Communication during aggressive interactions with particular reference to variation in choice of behavior . Animal Behavior, 33, 1152-1161 .
13.
Gould, J. L. , & Gould, C. G. (1999). The animal mind (pp. 105-113). New York: Scientific American Library .
14.
Hauser, M. D. (1996). The Evolution of communication. Cambridge, MA: MIT Press .
15.
Hunt, A. , & Thomas, D. (1999). The pragmatic programmer: From journeyman to master. Reading, MA: Addison-Wesley .
16.
Larman, C. , & Basili, V. R. (2003). Iterative and incremental development: A brief history . Computer, 36(6), 47-56 .
17.
Lyons, D. M. , & Arbib, M. A. (1989). A formal model of computation for sensory-based robotics . IEEE Transactions on Robotics and Automation, 5(3), 280-293 .
18.
Maes, P. , & Brooks, R. A. (1990). Learning to coordinate behaviours . In Proceedings of the Eighth National Conference on Artificial Intelligence (AAAI’90). Boston, MA. (pp. 796-802 ). Boston, MA: AAAI Press/MIT Press.
19.
Mahadevan, S. , & Connell, J. H. (1992). Automatic programming of behavior-based robots using reinforcement learning . Artificial Intelligence, 55(2-3), 304-312 .
20.
Mataric, M. J. (1997). Reinforcement learning in the multi-robot domain . Autonomous Robots, 4(1), 73-83 .
21.
Moore, B. R. (1996). The Evolution of Imitative Learning. In C. M. Heyes , & B. G. Galef (Eds.). Social learning in animals: The roots of culture (pp. 245-265). San Diego, CA: Academic Press .
22.
Nicolescu, M. , & Mataric, M. J. (2001). Experience-based representation construction: Learning from human and robot teachers . In Proceedings of the IEEE International Symposium on Computational Intelligence in Robots and Automation (CIRA’01). Banff, Canada. (pp. 463-468 ). IEEE.
23.
Nicolescu, M. , & Mataric, M. J. (2002). A hierarchical architecture for behavior-based robotics . In Proceedings of the 1st International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS’02) (pp. 227-233 ). Bologna, Italy: ACM.
24.
Nolfi, S. , & Floreano, D. (2000). Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. Cambridge, MA: MIT Press .
25.
O'Keefe, J. (1996). Geometric determinants of the place fields of hippocampal neurons . Nature, 381, 425-428 .
26.
Parker, L. E. (1998). ALLIANCE: An architecture for fault tolerant, cooperative control of heterogeneous mobile robots . IEEE Transactions on Robotics and Automation, 14(2), 220-240 .
27.
Pressman, R. S. (2001). Software engineering: A practitioners approach (5th ed.), (pp. 280-282). Boston, MA: McGraw-Hill .
28.
Raymond, E. S. (1999). The cathedral and the bazar. In E. S. Raymond (Ed.) The Cathedral and the bazar: Musings on Linux and Open Source by an accidental revolutionary (pp. 19-64). Sebastopol, CA: O'Reilly .
29.
Stone, P. , & Veloso, M. (1999). Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork . Artificial Intelligence, 110(2), 241-273 .
30.
Weng, J. , McClelland, J. , Pentland, A. , Sporns, O. , Stockman, I. , Sur, M. , & Thelen, E. (2000). Autonomous mental development by robots and animals . Science, 291(5504), 599-600 .
31.
Werger, B. B. (2000). Ayllu: Distributed port-arbitrated behavior-based control . In L. E. Parker , G. Bekey , & J. Barhen (Eds.), Distributed Autonomous Robotic Systems 4, Proceedings of the Fifth International Symposium on Distributed, Autonomous Robotic Systems (DARS’00). Knoxville, TN (pp. 25-34 ). Springer.
