The adaptive value of emotions in nature indicates that they might also be useful in artificial creatures. Experiments were carried out to investigate this hypothesis in a simulated learning robot. For this purpose, a non-symbolic emotion model was developed that takes the form of a recurrent artificial neural network where emotions both depend on and influence the perception of the state of the world. This emotion model was integrated in a reinforcement-learning architecture with three different roles: influencing perception, providing reinforcement value, and determining when to reevaluate decisions. Experiments to test and compare this emotion-dependent architecture with a more conventional architecture were done in the context of a solitary learning robot performing a survival task. This research led to the conclusion that artificial emotions are a useful construct to have in the domain of behavior-based autonomous agents with multiple goals and faced with an unstructured environment, because they provide a unifying way to tackle different issues of control, analogous to natural systems' emotions.
Albus, J. S. (1990). The role of world modeling and value judgment in perception. In A. Meystel, J. Herath, & S. Gray (Eds.), Proceedings of the 5th IEEE International Symposium on Intelligent Control. Los Alamitos, CA: IEEE Computer Society Press.
2.
Araujo, A. F. R. (1994). Memory, emotions & neural networks. Unpublished doctoral dissertation, Sussex University.
3.
Asada, M. (1996). An agent and an environment: A view on body scheme. In J. Tani & M. Asada (Eds.), Proceedings of the 1996 IROS Workshop on Towards Real Autonomy(pp. 19–24). Osaka: Senri Life Science Center.
4.
Aubé, M. (1998). A commitment theory of emotions. In D. Cañamero (Ed.), AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition(Tech. Rep. No. FS-98-03, pp. 13–18). Menlo Park, CA: AAAI Press.
5.
Bates, J. (1994). The role of emotions in believable agents (Tech. Rep. No. CMU-CS-94-136). Pittsburgh, PA: Carnegie Mellon University, School of Computer Science.
6.
Bates, J., Loyall, A. B., & Reilly, W. S. (1992a). An architecture for action, emotion, and social behavior. In Artificial social systems: Fourth European Workshop on Modeling Autonomous Agents in a Multi-Agent World. Berlin: Springer. (Also available as Tech. Rep. No. CMU-CS-92-144. Pittsburgh, PA: Carnegie Mellon University, School of Computer Science).
7.
Bates, J., Loyall, A. B., & Reilly, W. S. (1992b). Integrating reactivity, goals, and emotion in a broad agent (Tech. Rep. No. CMU-CS-92-142). Pittsburgh, PA: Carnegie Mellon University, School of Computer Science.
8.
Beaudoin, L., & Sloman, A. (1993). A study of motive processing and attention. In A. Sloman, D. Hogg, G. Humphreys, A. Ramsay, & D. Partridge (Eds.), Proceedings of AISB93(pp. 229–238). Oxford: IOS Press.
9.
Bechara, A., Damasio, H., Tranel, D., & Damásio, A. R. (1997). Deciding advantageously before knowing the advantageous stragegy. Science, 275, 1293–1295.
10.
Beck, R. C. (1983). Motivation: Theories and principles(2nd ed.). Engelwood Cliffs, NJ: Prentice-Hall.
11.
Blaney, P. (1986). Affect and memory. Psychological Bulletin, 99, 229–246.
12.
Blumberg, B. (1996). Old Tricks, new dogs: Ethology and interactive creatures. Unpublished doctoral dissertation Massachusetts Institute of Technology.
13.
Bower, G. H. (1981). Mood and memory. American Psychologist, 36, 129–148.
14.
Bower, G. H., & Cohen, P. R. (1982). Emotional influences in memory and thinking: Data and theory. In M. S. Clark & S. T. Fisk (Eds.), Affect and cognition—The Seventeenth Annual Carnegie Symposium on Cognition(pp. 291–331). Hillsdale, NJ: Erlbaum.
15.
Bozinovski, S. (1982). A self-learning system using secondary reinforcement. In R. Trappl (Ed.), Cybernetics and systems(pp. 397–402). North-Holland: Elsevier.
16.
Breazeal, C. (1998). Early experiments using motivations to regulate human-robot interaction. In D. Cañamero (Ed.), AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition(Tech. Rep. No. FS-98-03, pp. 31–36). Menlo Park, CA: AAAI Press.
17.
Cañamero, D. (1997). Modeling motivations and emotions as a basis for intelligent behavior. In D. Cañamero (Ed.), Proceedings of the First International Symposium on Autonomous Agents, AA'97. Menlo Park, CA: ACM Press.
18.
Cañamero, D. (1998). Issues in the design of emotional agents. In D. Cañamero (Ed.), AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition(Tech. Rep. No. FS-98-03, pp. 49–54). Menlo Park, CA: AAAI Press.
19.
Charland, L. C. (1995). Emotion as a natural kind: Towards a computational foundation for emotion theory. Philosophical Psychology, 8, 59–84.
20.
Cytowic, R. E. (1993). The man who tasted shapes. London: Abacus.
21.
Damásio, A. R. (1994). Descartes' error—Emotion, reason and human brain. London: Picador.
22.
Darwin, C. (1965). The expression of the emotions in man and animals. Chicago: University of Chicago Press. (Original work published in 1872).
23.
Derryberry, D., & Tucker, D. M. (1994). Motivating the focus of attention. In P. M. Niedenthal, & S. Kitayama (Eds.), The heart's eye(pp. 167–196). New York: Academic Press.
24.
De Sousa, R. (1987). The rationality of emotion. Cambridge, MA: MIT Press.
25.
Dreyfus, H. L. (1992). What computers still can't do: A critique of artificial reason. Cambridge, MA: MIT Press.
26.
Dyer, M. G. (1987). Emotions and their computations: Three computer models. Cognition and Emotion, 1, 323–347.
27.
Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6, 169–200.
28.
Foliot, G., & Michel, O. (1998). Learning object significance with an emotion based process. In Workshop W5: Grounding Emotions in Adaptive Systems (pp. 25–30). Workshop conducted at SAB'98: Fifth International Conference on Simulation of Adaptive Behavior.
29.
Frijda, N. H., & Swagerman, J. (1987). Can computers feel? Theory and design of an emotional system. Cognition and Emotion, 1(3), 235–257.
30.
Gadanho, S. C. (1999). Reinforcement learning in autonomous robots: An empirical investigation of the role of emotions.Unpublished doctoral dissertation, University of Edinburgh.
31.
Gadanho, S. C., & Hallam, J. (1998). Emotion-driven learning for animat control. In R. Pfeifer, B. Blumberg, J. A. Meyer, & S. W. Wilson (Eds.), From animals to animals 5— Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior(pp. 354–359). Cambridge, MA: MIT Press.
32.
Gadanho, S. C., & Hallam, J. (2001). Emotion-triggered learning in autonomous robot control. In D. Cañamero, C. Numaoka, & P. Petta (Eds.), Grounding emotions in adaptive systems—A special issue of Cybernetics and Systems, 32(5), 531–559.
33.
Goleman, D. (1995). Emotional intelligence. London: Bloomsbury Publishing.
34.
Grossberg, S., & Gutowski, W. (1987). Neural dynamics of decision making under risk: Affective balance and cognitiveemotional interactions. Psychological Review, 94, 300–318.
35.
Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neutral computation. Reading, MA: Addison-Wesley.
36.
Izard, C. E. (1993). Four systems for emotion activation: Cognitive and noncognitive processes. Psychological Review, 100, 68–90.
37.
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237–285.
38.
Kaiser, S., & Wehrle, T. (1996). Situated emotional problem solving in interactive computergames. In N. H. Frijda (Ed.), Proceedings of the 8th Conference of the International Society for Research on Emotions, ISRE'96(pp. 276–280). Storrs, CT: ISRE Publications.
39.
Kitano, H. (1995). A model for hormonal modulation of learning. In C. Mellish (Ed.), IJCAI-95— Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence(Vol. 1, pp. 532–538). San Francisco: Morgan Kaufmann.
40.
Klein, J. T. (1996). Computer response to frustration. Master's thesis, Massachusetts Institute of Technology, School of Architecture and Planning.
41.
Lazarus, R. S. (1982). Thoughts on the relations between emotion and cognition. American Psychologist, 37, 1019–1024.
42.
LeDoux, J. E. (1998). The emotional brain. London: Phoenix.
43.
Lin, L.-J. (1992). Self-improving reactive agents based on reinforcement learning planning and teaching. Machine Learning, 8, 293–321.
44.
Lin, L.-J. (1993). Reinforcement learning for robots using neural networks. Unpublished doctoral dissertation, Carnegie Mellon University. (Tech. Rep. No. CMU-CS-93–103).
45.
Mahadevan, S., & Connell, J. (1992). Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55, 311–365.
46.
Mataric, M. J. (1994). Reward functions for accelerated learning. In W. W. Cohen, & H. Hirsh (Eds.), Machine learning: Proceedings of the Eleventh International Conference(pp. 181–189). San Francisco, CA: Morgan Kaufmann.
47.
McCauley, L., & Franklin, S. (1998). An architecture for emotion. In D. Cañamero (Ed.), AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition(Tech. Rep. No. FS-98-03, pp. 122–127). Menlo Park, CA: AAAI Press.
48.
Michaud, F., Lachiver, G., & Dinh, C. T. L. (1996). A new control architecture combining reactivity, planning, deliberation and motivation for a situated autonomous agent. In P. Maes, M. J. Mataric, J.-A. Meyer, J. Pollack, & S. W. Wilson (Eds.), From animals to animals 4—Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior(pp. 245–254). Cambridge, MA: MIT Press.
49.
Michel, O. (1996). Khepera Simulatorpackage version 2.0: Freeware mobile robot simulator written at the University of Nice Sophia–Antipolis. Downloadable from the World Wide Web at http://wwwi3s.unice.fr/om/khep-sim.html.
50.
Minsky, M. (1986). The society of mind. New York: Simon and Schuster.
51.
Morignot, P., & Hayes-Roth, B. (1995). Why does an agent act? In M. Cox & M. Freed (Eds.), AAAI Spring Symposium on Representing Mental States and Mechanisms(pp. 97–101). Menlo Park, CA: AAAI Press. (Also available as report KSL 95–76 of Stanford University).
52.
Mowrer, O. H. (1960). Learning theory and behavior. New York: Wiley.
53.
Niedenthal, P. M., Setterlund, M. B., & Jones, D. E. (1994). Emotional organization of perceptual memory. In P. M. Niedenthal & S. Kitayama (Eds.), The heart's eye(pp. 87–113). New York: Academic Press.
54.
Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press.
55.
Panksepp, J. (1995). The emotional brain and biological psychiatry. Advances in Biological Psychiatry, 1, 263–288.
56.
Pfeifer, R. (1982). Cognition and emotion: An information process approach (CIP Working Paper 436). Pittsburgh, PA: Carnegie-Mellon University, Department of Psychology.
57.
Pfeifer, R., & Nicholas, D. W. (1985). Toward computational models of emotion. In L. Steels & J. A. Campbell (Eds.), Progress in artificial intelligence(pp. 184–192). Chichester, UK: Ellis Horwood.
58.
Picard, R. (1997). Affective computing. Cambridge, MA: MIT Press.
59.
Power, M., & Dalgleish, T. (1997). Cognition and emotion. Philadelphia: Psychology Press.
60.
Rodriguez, M., & Muller, J.-P. (1995). Towards autonomous cognitive animats. In F. Morán, A. Moreno, J. Merelo, & P. Chacón, (Eds.), Advances in artificial life — Proceedings of the Third European Conference on Artificial Life. Lecture Notes in Artificial Intelligence Vol. 929. Berlin: Springer-verlag.
61.
Seif EI-Nasr, M., Ioerger, T. R., & Yen, J. (1998). Learning and emotional intelligence in agents. In D. Cañamero (Ed.), AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition(Tech. Rep. No. FS-98-03, pp. 150–155). Menlo Park, CA: AAAI Press.
62.
Shaver, P., Schwartz, J., Kirson, D., & O'Connor, C. (1987). Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology, 52, 1061–1086.
63.
Shibata, T., Ohkawa, K., & Tanie, K. (1996). Spontaneous behavior of robots for cooperation—Emotionally intelligent robot system. In Proceedings of IEEE International Conference on Robotics and Automation(pp. 2426–2431). Tokyo, Japan.
64.
Simon, H. A. (1967). Motivational and emotional controls of cognition. Psychological Review, 74, 29–39.
65.
Sloman, A., Beaudoin, L., & Wright, I. (1994). Computational modeling of motive-management processes. In N. Frijda (Ed.), Proceedings of the Conference of the International Society for Research in Emotions(pp. 344–348). Cambridge, UK: ISRE Publications.
66.
Sloman, A., & Croucher, M. (1981). Why robots will have emotions. In IJCAI'81— Proceedings of the Seventh International Joint Conference on Artificial Intelligence(pp. 2369–2371).
67.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.
68.
Toda, M. (1993). The urge theory of emotion and cognition: Chapter 1 Emotions and urges (SCCS Tech. Rep. No. 93-1-01). Chuyko University.
69.
Toda, M. (1994). Emotion, society and the versatile architecture (SCCS Tech. Rep. No. 94-1-02). Chuyko University.
70.
Tomkins, S. S. (1984). Affect theory. In K. R. Scherer, & P. Ekman (Eds.), Approaches to emotion. London: Erlbaum.
71.
Velásquez, J. D. (1998). A computational framework for emotion-based control. In Workshop W5: Grounding Emotions in Adaptive Systems(pp. 62–67). Workshop conducted at SAB'98: Fifth International Conference on Simulation of Adaptive Behavior.
72.
Ventura, R., Custódio, L., & Pinto-Ferreira, C. (1998). Emotions—the missing link? In D. Cañamero (Ed.), AAAI Fall Symposium on Emotional and Intelligent: The tangled knot of cognition(Tech. Rep. No. FS-98-03, pp 170–175). Menlo Park, CA: AAAI Press.
73.
Watkins, C. (1989). Learning from delayed rewards. Unpublished doctoral dissertation, King's College, Cambridge.
74.
Wehrle, T. (1998). Motivations behind modeling emotions agents: Whose emotions does your robot have? In Workshop W5: Grounding Emotions in Adaptive Systems(pp. 71–76). Workshop conducted at SAB'98: Fifth International Conference on Simulation of Adaptive Behavior.
75.
Wilson, S. W. (1991). The animat path to AI. In J.-A. Meyer & S. W. Wilson (Eds.), From animals to animats— Proceedings of the First Conference on Simulation of Adaptive Behavior(pp. 15–21). Cambridge, MA: MIT Press.
76.
Wright, I. (1996). Reinforcement learning and animat emotions. In P. Maes, M. J. Mataric, J.-A. Meyer, J. Pollack, & S. W. Wilson (Eds.), From animals to animats 4— Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior(pp. 273–281). Cambridge, MA: MIT Press.
77.
Zajonc, R. B. (1984). On primacy of affect. In K. R. Scherer & P. Ekman, (Eds.), Approaches to emotion. London: Erlbaum.
