We explore the use of behavior-based architectures within the context of reinforcement learning and examine the effects of using different behavior-based architectures on the ability to learn correctly and efficiently the task at hand. In particular, we study the task of learning to push boxes in a simulated two-dimensional environment originally proposed by Mahadevan and Connell (1992). We examine issues such as effectiveness of learning, flexibility of the learning method to adapt to new environments, and effect of the behavior architecture on the ability to learn, and we report results obtained on a large number of simulation runs.
Brooks, R. (1986). A robust layered control system for a mobile robot . IEEE Journal of Robotics and Automation, RA-2(1), 14-23.
2.
Comer, D. (1979). The ubiquitous B tree. ACM Computing Surveys, 11(2), 121-137.
3.
Connell, J., & Mahadevan, S. (1993). Rapid task learning for real robots. In J. H. Connell & S. Mahadevan (Eds.), Robot learning. Norwell, MA : Kluwer Academic.
4.
Dorigo, M. (1995). Alecsys and the autonomouse: Learning to control a real robot by distributed classifier systems. Machine Learning , 19(3), 209-240.
5.
Dorigo, M., & Colombetti, M. (1994). Robot shaping: Developing autonomous agents through learning. Artificial Intelligence , 71(2), 321-370.
Hougen, D.E., Fischer, J., Gini, M., & Slagle, J. (1996). Fast connectionist learning for trailer backing using a real robot. In Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, MN. Los Alamitos, CA: IEEE Press.
8.
Kaelbling, L.P., Littman, M.L., & Moore, A.W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.
9.
Koza, J.R., & Rice, J.P. (1992). Automatic programming of robots using genetic programming. In Proceedings of the National Conference on Artificial Intelligence , San José, CA. Menlo Park, CA: AAAI Press.
10.
Mahadevan, S. (1994). To discount or not to discount in reinforcement learning: A case study comparing R-learning and Q-learning. In Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, NJ. San Mateo; CA: Morgan Kaufmann.
11.
Mahadevan, S. (1996). Average reward reinforcement learning: Foundations, algorithms, and empirical results. Machine Learning, 22, 159-196.
12.
Mahadevan, S., & Connell, J. (1992). Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55, 311-365.
13.
Mataric, M. (1994). Reward functions for accelerated learning. In Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, NJ. San Mateo, CA: Morgan Kaufmann.
14.
Schwartz, A. (1993). A reinforcement learning method for maximizing undiscounted rewards. In Proceedings of the Tenth International Conference on Machine LearningSan Mateo, CA: Morgan Kaufmann.
15.
Singh, S.P. (1992). Transfer of learning by composing solutions of elemental sequential tasks. Machine Learning, 8(3-4), 323-339.
Whitehead, S.D. (1992). Reinforcement learning for the adaptive control of perception and action. Unpublished doctoral thesis, University of Rochester, Rochester, NY
19.
Whitehead, S.D., Karlsson, J., & Tenenberg, J. (1993). Learning multiple goal behavior via task decomposition and dynamic policy merging. In J. H. Connell and S. Mahadevan (Eds.), Robot learning. Norwell, MA: Kluwer Academic.
