This article introduces a formalization of the dynamics between sensorimotor interaction and homeostasis, integrated in a single architecture to learn object affordances of consummatory behaviors. We also describe the principles necessary to learn grounded knowledge in the context of an agent and its surrounding environment, which we use to investigate the constraints imposed by the agent’s internal dynamics and the environment. This is tested with an embodied, situated robot, in a simulated environment, yielding results that support this formalization. Furthermore, we show that this methodology allows learned affordances to be dynamically redefined, depending on object similarity, resource availability, and the rhythms of the agent’s internal physiology. For example, if a resource becomes increasingly scarce, the value assigned by the agent to its related effect increases accordingly, encouraging a more active behavioral strategy to maintain physiological stability. Experimental results also suggest that a combination of motivation-driven and affordance learning in a single architecture should simplify its overall complexity while increasing its adaptivity.
Ackley, D., & Littman, M. ( 1991). Interactions between learning and evolution. In C. Langton, C. Taylor, J. D. Farmer, & S. Rasmussen (Eds.), Artificial life II, SFI studies in the sciences of complexity (Vol. X, pp. 487-509). Menlo Park, CA: Addison-Wesley.
2.
Ashby, W. ( 1965). Design for a brain: The origin of adaptive behaviour . London: Chapman & Hall.
3.
Avila-García, O., & Cañamero, L. (2002). A comparison of behaviour selection architectures using viability indicators. In Proceedings of International Workshop on Biologically Inspired Robotics: The Legacy of W. Grey Walter. Bristol HP Labs, UK.
4.
Baecker, R., Grudin, J., Buxton, W.A., & Greenberg, S. ( 1995). Readings in human-computer interaction: Towards the year 2000. San Francisco, CA: Morgan Kaufmann.
5.
Bindra, D. ( 1969). The interrelated mechanisms of reinforcement and motivation, and the nature of their influence on response. In W. J. Arnold & D. Levine (Eds.), Nebraska symposium on motivation (pp. 1-33). University of Nebraska Press.
6.
Cannon, W.B. ( 1929). The wisdom of the body. London: Kegan Paul, Trench, Trubner & Co.
7.
Chemero, A., & Turvey, M.T. ( 2007). Gibsonian affordances for roboticists. Adaptive Behavior, 15(4), 473-480.
8.
Cisek, P. ( 2007). Cortical mechanisms of action selection: The affordance competition hypothesis. Philosophical Transactions of the Royal Society B, 362, 1585-1599.
9.
Cisek, P., & Kalaska, J. ( 2005). Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron, 45(5), 801-814.
10.
Cooper, R., & Glasspool, D. ( 2002). Learning affordances and action schemas. In R. French & J. Sougne (Eds.), Connectionist models of learning, development and evolution (pp. 133-142). London: Springer-Verlag .
11.
Cos-Aguilera, I., Cañamero, L., & Hayes, G. ( 2003a). Motivation-driven learning of object affordances: First experiments using a simulated khepera robot. In F. Detje, D. Dörner, & H. Schaub (Eds.), The logic of cognitive systems. Proceedings of the 5th International Conference on Cognitive Modelling, ICCM (pp. 57-62). Universitäts-Verlag Bamberg.
12.
Cos-Aguilera, I., Cañamero, L., & Hayes, G.M. ( 2003b). Learning object functionalities in the context of behavior selection. In U. Nehmzow & C. Melhuish (Eds.), in Proceedings of the 3rd Conference Towards Intelligent Mobile Robotics. (pp. 9-14)
13.
Cos-Aguilera, I., Cañamero, L., & Hayes, G.M. ( 2004). Using a sofm to learn object affordances. In J. L. De la Rosa & J. A. Ramón-Pagés (Eds.), in Proceedings of the 5th Workshop of Physical Agents (WAF04). (pp. 139-150). University of Girona.
14.
Damoulas, T., Cos-Aguilera, I., Hayes, G., & Taylor, T. ( 2005). Valency for adaptive homeostatic agents: Relating evolution and learning. In Proceedings of the 8th European Conference on Artificial Life (ECAL2005), Lecture Notes in Artificial Intelligence (Vol. 3630, pp. 936-945). Berlin/ Heidelberg: Springer.
15.
Doğar, M., Ugur, E., Şahin, E., & Çakmak, M. (2008). Using learned affordances for robotic behavior development. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation. (pp. 3802-3807). Pasadena, CA: IEEE
16.
Duchon, A.P., & Warren, W.H. ( 1994). Robot navigation from a Gibsonian viewpoint. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics (pp. 2-5). IEEE.
17.
Duchon, A.P., Warren, W.H., & Kaelbling, L.P. (1995). Ecological robotics: Controlling behaviour with optical flow. In J. Moore & J. Lehman (Eds.), Proceedings of the 17th Annual Cognitive Science Conference. (pp. 164-169). Mahwah, NJ: Lawrence Erlbaum Associates.
18.
Fagg, A.H., & Arbib, M.A. ( 1998). Modeling parietal-premotor interactions in primate control of grasping. Neural Networks, 11(7-8), 1277-1303.
19.
Fellous, J.-M. ( 2001). Dopamine modulation of prefrontal delay activity-reverberatory activity and sharpness of tuning curves. Neurocomputing, 38-40, 1549-1556.
20.
Fellous, J.-M. ( 2004). From human emotions to robotic emotions. In L. Cañamero & E. Hudlika (Eds.), Architectures for modelling emotion: Cross-disciplinary foundations. Papers from the 2004 AAAI Spring Symposium . Menlo Park, CA: AAAI Press.
21.
Fellous, J.-M., & Suri, R.E. ( 2003). The handbook of brain theory and neural networks (chap. The Roles of Dopamine). Cambridge, MA: MIT Press.
22.
Freud, S. ( 1940). An outline of psychoanalysis. London : Hogarth Press.
23.
Gibson, J.J. ( 1986). The ecological approach to visual perception. Hillsdale, N.J./London.
24.
Harnad, S. ( 1990). The symbol grounding problem. Physica D, 42, 335-346.
25.
Hebb, D. ( 1949). The organization of behavior. A neuropsychological theory . New York: Wiley.
26.
Hinde, R.A. ( 1960). Energy models of motivation. In Symposium of Society of Experimental Biology (Vol. 14, pp. 199-213). Cambridge University Press.
27.
Houk, J.C., Adams, J.L., & Barto, A.G. ( 1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (2nd ed., 1998; pp. 249-270). Cambridge, MA: MIT Press/Bradford Books,
28.
Hull, C. ( 1943). Principles of behaviour: an introduction to behaviour theory. New York, NY: D. Appleton-Century Company, Inc.
29.
Khamassi, M., Lachèze, L., Girard, B., & Guillot, A. ( 2006). Actor-critic models of reinforcement learning in the basal ganglia: From natural to artificial rats. Adaptive Behavior , 13(2), 131-148.
30.
Konidaris, G., & Barto, A. ( 2006). An adaptive robot motivational system. In S. Nolfi, G. Baldassarre , R. Calabretta, J. C. T. Hallam, D. Marocco, J.-A. Meyer, O. Miglino , & D. Parisi (Eds.), Animals to animats 9: Proceedings of the 9th International Conference on Simulation of Adaptive Behavior (pp. 346-356). Berlin: Springer Verlag.
31.
Konidaris, G., & Hayes, G.M. ( 2005). An architecture for behavior-based reinforcement learning . Adaptive Behavior. 69, 861-872.
32.
Krieckhaus, E., & Wolf, G. ( 1968). Acquisition sodium by rats: Interaction of innate mechanisms and latent learning. Journal of comparative and Physiological Psychology, 65, 197-201.
33.
Lorenz, K. ( 1966). Evolution and modification of behaviour. London: Methuen & Co Ltd.
34.
Marrow, A. ( 1969). The practical theorist: The life and work of Kurt Levin . New York: Basic Books.
35.
Marsland, S., Shapiro, J., & Nehmzow, U. ( 2002). A selforganising neural network that grows when required . Neural Networks, 15(8-9), 1041-1058.
36.
Matarić, M., & Brooks, R. ( 1990). Cambrian intelligence: The early history of the new AI (chap. Learning a distributed map representation based on navigation behaviors) . Cambridge, MA: MIT Press.
37.
Maturana, H.R., & Varela, F.J. ( 1980). Boston Studies in the Philosophy of Science: Vol. 42. Autopoiesis and cognition: The realization of the living. Dordrecht/London: Reidel.
38.
McDougall, W. ( 1913). The sources and direction of psychophysical energy. American Journal of Insanity.
McFarland, D.J., & Sibly, R.M. ( 1975). The behavioural final common path. Proceedings of the Royal Society of London, 270, 265-293.
41.
Morineau, T., & Frénod, E. (2009). Affordance modelling and simulation in virtual environment. European Journal of Scientific Research, 33(1), 130-143.
42.
Nehaniv, C., & Dautenhahn, K. (1998). Mapping between dissimilar bodies: Affordances and the algebraic foundations of imitation. In J. Demiris & A. Birk (Eds.), Proceedings of the European Workshop on Learning Robots 1998 (EWLR-7) (pp. 177-184) Edinburgh, UK.
43.
Redgrave, P., Prescott, T., & Gurney, K. ( 1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience, 89, 1009-1023.
44.
Şahin, E., Çakmak, M., Doğar, M.R., Ugur, E., & Üçoluk, G. (2007). To afford or not to afford: A new formalization of affordances toward affordance-based robot control. Adaptive Behavior, 15, 447-472.
45.
Song, J., & Nakayama, K. ( 2007). Automatic adjustment of visuomotor readiness. Journal of Vision, 7(5), 1-9.
46.
Spier, E., & McFarland, D. ( 1996). A finer-grained motivational model of behaviour sequencing . In P. Maes, M. J. Mataric, J.-A. Meyer, J. Pollack, & S. W. Wilson (Eds.), A Finer-Grained Motivational Model of Behaviour Sequencing. From Animals to Animats 4: Proceedings of SAB96 (pp. 255-263). Cambridge, MA: MIT Press.
47.
Spier, E., & McFarland, D. ( 1997). Possibly optimal decision-making under self-sufficiency and autonomy. Journal of Theoretical Biology, 189, 317-331.
48.
St. Amant, R. ( 1999). User interface affordances in a planning representation . Human-Computer Interaction, 14(3), 317-354.
49.
Stoffregen, T. ( 2003). Affordances as properties of the animal environment system . Ecological Psychology, 15, 115-134. Sutton, R., & Barto, A. (1981). Towards a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 198(88), 135-170.
50.
Tinbergen, N. ( 1951). The study of instinct. Oxford: Oxford University Press.
51.
Turvey, M. ( 1992). Affordances and prospective control: an outline of the ontology. Ecological Psychology, 4, 271- 281.
52.
Usher, M., & Davelaar, E.J. (2002). Neuromodulation of decision and response selection. Neural Networks, 15, 635- 645.
53.
Widrow, B., & Hoff, M. ( 1960). Adaptive switching circuits. In 1960 IRE WESCON Convention Record (pt. 4) (pp. 96- 104). IRE
