In this article, we present a learning mechanism that allows a multiagent system to cooperate to achieve a gathering task efficiently in unknown and changing environments. The multiagent system is a team of autonomous behavior-based agents with limited communication capabilities. Cooperation is based on the acquisition of signaling behaviors and on the specialization of the agents into different types. Every agent has the same collection of built-in reactive behaviors. Some of the built-in behaviors are fixed, whereas others can be modified through reinforcement learning. The reinforcement signal is delayed until a trial is completed and assesses the collective performance of the team. Each agent uses this common signal to learn what individual behaviors are more suitable for the team. Simulation results, and the corresponding statistical analysis, show that the multiagent system always achieves near-optimal performances.
Arkin, R.C. (1989). Motor schema-based mobile robot navigation. International Journal of Robotics Research, 8, 92-112.
2.
Barto, A.G., Bradtke, S.J., & Singh, S.P. (1995). Learning to act using real-time dynamic programming . Artificial Intelligence, 72, 81-138.
3.
Barto, A.G., Sutton, R.S., & Watkins, C.J.C.H. (1989). Learning and sequential decision making (Tech. Rep. COINS-89-95). Amherst: Department of Computer and Information Science, University of Massachusetts .
4.
Drogoul, A., & Ferber, J. (1993). From Tom Thumb to the Dockers: Some experiments with foraging robots. In J.-A. Meyer, H. L. Roitblat, & S. W Wilson (Eds.), From animals to animats II: Proceedings of the Second International Conference on Simulation of Adaptive Behavior . Cambridge, MA: MIT Press.
5.
Goss, S., & Deneubourg, J.L. (1992). Harvesting by a group of robots. In F.J. Varela & P. Bourgine (Eds.), Toward a practice of autonomous systems: Proceedings of the First European Conference on Artificial Life. Cambridge, MA: MIT Press.
6.
Kube, C.R., & Zhang, H. (1993). Collective robotics: From social insects to robots . Adaptive Behavior, 2, 189-218.
7.
Lin, L.-J. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning, 8, 293-321.
8.
Mahadevan, S., & Connell, J. (1992). Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55, 311-365.
9.
Matarić, M.J. (1993). Designing emergent behaviors: From local interactions to collective intelligence. In J.-A. Meyer , H. L. Roitblat, & S. W Wilson (Eds.), From animals to animats II: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press .
10.
Matarić, M.J. (1994a). Reward functions for accelerated learning. In W. W. Cohen & H. Hirsh (Eds.), Machine learning: Proceedings of the Eleventh International Conference. San Mateo, CA: Morgan Kaufmann.
11.
Matarić, M.J. (1994b). Learning to behave socially. In D. Cliff, P. Husbands, J.-A. Meyer, & S. W Wilson (Eds.), From animals to animats III: Third International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press.
12.
Millán, J. del R. (1994). Learning efficient reactive behavioral sequences from basic reflexes in a goal-directed autonomous robot. In D. Cliff, P Husbands, J.-A. Meyer, & S. W Wilson (Eds.), From animals to animats III: Third International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press.
13.
Millán, J. del R., & Torras, C. (1992). A reinforcement connectionist approach to robot path finding in non-maze-like environments. Machine Learning , 8, 363-395.
14.
Mitchell, T.M., & Thrun, S.B. (1993). Explanation-based neural networks learning for robot control. In C. L. Giles, S. J. Hanson, & J. D. Cowan (Eds.), Advances in neural information processing systems 5. San Mateo, CA: Morgan Kaufmann.
15.
Murciano, A. (1995). Aprendizaje de comportamientos cooperativos en sociedades de agentes autónomos. Unpublished doctoral thesis, Departmento de Biomatemática, Universidad Complutense de Madrid.
16.
Numaoka, C., & Takeuchi, A. (1993). Collective choice of strategic type. In J.-A. Meyer, H. L. Roitblat , & S. W Wilson (Eds.), From animals to animats II: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press.
17.
Prescott, T.J., & Mayhew, J. E. W (1992). Obstacle avoidance through reinforcement learning. In J. E. Moody , S. J. Hanson, & R. P. Lippmann (Eds.), Advances in Neural Information Processing Systems 4. San Mateo, CA: Morgan Kaufmann.
18.
Steels, L. (1994). A case study in the behavior-oriented design of autonomous agents. In D. Cliff, P Husbands, J.-A. Meyer, & S. W Wilson (Eds.), From animals to animats III: Third International Conference on Simulation of Adaptive Behavior . Cambridge, MA: MIT Press.
19.
Sutton, R.S. (1984). Temporal credit assignment in reinforcement learning. Unpublished doctoral thesis, Department of Computer and Information Science, University of Massachusetts, Amherst.
20.
Sutton, R.S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. Proceedings of the Seventh International Conference on Machine LearningSan Mateo, CA: Morgan Kaufmann.
21.
Tan, M. ( 1993). Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the Tenth International Conference on Machine LearningSan Mateo, CA: Morgan Kaufmann .
22.
Weiss, G. (1993). Action selection and learning in multi-agent environments . In J.-A. Meyer, H. L. Roitblat, & S. W. Wilson (Eds.), From animals to animats II: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press.
23.
Yanco, H., & Stein, L.A. (1993). An adaptive communication protocol for cooperating mobile robots. In J.-A. Meyer, H. L. Roitblat, & S. W Wilson (Eds.), From animals to animats II: Proceedings of the Second International Conference on Simulation of Adaptive Behavior . Cambridge, MA: MIT Press.
