One of the main assertions of sensorimotor contingency theory is that sensory experience is not generated by activating an internal representation of the outside world through sensory signals, but corresponds to a mode of exploration and hence is an active process. Perception and sensory awareness emerge from using the structure of changes in the sensory input resulting from these exploratory actions, called sensorimotor contingencies (SMCs), for planning, reasoning, and goal achievement. Using a previously developed computational model of SMCs we show how an artificial agent can plan ahead with SMCs and use them for action guidance. Our main assumption is that SMCs are associated with a utility for the agent, and that the agent selects actions that maximize this utility. We analyze the properties of the resulting actions in a robot that is endowed with several sensory modalities and controlled by our model in a simple environment. The results demonstrate that its actions avoid aversive events, and that it can achieve a low-level form of spatial awareness that is resilient to the complete loss of a sensory modality.
BernsteinN. (1967). The co-ordination and regulation of movements. Oxford, England: Pergamon Press.
2.
Bolado-GomezR.GurneyK. (2013). A biologically plausible embodied model of action discovery. Frontiers in Neurorobotics, 7, 4.
3.
BootsB.SiddiqiS.GordonG. (2011). Closing the learning-planning loop with predictive state representations. The International Journal of Robotics Research, 30(7), 954–966.
4.
BotvinickM. M. (2007). Multilevel structure in behaviour and in the brain: a model of Fuster’s hierarchy. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1615–1626.
5.
BovetS.PfeiferR. (2005). Emergence of delayed reward learning from sensorimotor coordination. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 2272–2277).
6.
ChenL. L.LeeD.FukushimaK.FukushimaJ. (2012). Submovement composition of head movement. PLoS ONE7(11), e47565.
7.
ChoeY.YangH.-F.EngD.-Y. (2007). Autonomous learning of the semantics of internal sensory states based on motor exploration. International Journal of Humanoid Robotics, 4, 211–243.
8.
CisekP. (2007). Cortical mechanisms of action selection: the affordance competition hypothesis. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1585–1599.
9.
CisekP.KalaskaJ. F. (2010). Neural mechanisms for interacting with a world full of action choices. Annual Review of Neuroscience, 33(1), 269–298.
10.
CrabbeF. L. (2007). Compromise strategies for action selection. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1559–1571.
11.
DogarM.UgurE.SahinE.CakmakM. (2008). Using learned affordances for robotic behavior development. In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on (3802–3807).
12.
DrescherG. L. (1991). Made-Up Minds: A Constructivist Approach to Artificial Intelligence. Cambridge: MIT Press.
13.
FineP.Di PaoloE.IzquierdoE. (2007). Adapting to your body. In Ninth European Conference on Artificial Life (pp. 203–212). Springer.
14.
GawthropP.LoramI.LakieM.GolleeH. (2011). Intermittent control: a computational theory of human control. Biological Cybernetics, 104(1-2), 31–51.
15.
GibsonJ. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.
16.
HoffmannM.SchmidtN.PfeiferR.EngelA. K.MayeA. (2012). Using Sensorimotor Contingencies for Terrain Discrimination and Adaptive Walking Behavior in the Quadruped Robot Puppy. In ZiemkeT.BalkeniusC.HallamJ. (Eds.), From Animals to Animats 12 (7426, 54–64). Berlin, Heidelberg: Springer.
17.
HolmesM. P.IsbellC. L.Jr. (2005). Schema learning: experience-based construction of predictive action models. Advances in Neural Information Processing Systems, 17, 585–592.
18.
HowardR. (1977). Risk preference. In HowardR.MathesonJ. E.MillerK. L. (Eds.), Readings in Decision Analysis (pp. 431–465). Menlo Park, California: SRI International.
19.
JamesM.SinghS.LittmanM. (2004). Planning with predictive state representations. In Machine Learning and Applications, 2004. Proceedings. 2004 International Conference on (pp. 304–311).
20.
KaelblingL. P.LittmanM. L.CassandraA. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101 (1–2), 99–134.
21.
LittmanM.SuttonR.SinghS. (2002). Predictive representations of state. Advances in Neural Information Processing Systems, 14, 1555–1561.
22.
MayeA.EngelA. (2011). A discrete computational model of sensorimotor contingencies for object perception and control of behavior. In Robotics and Automation, 2011. ICRA 2011. IEEE International Conference on (3810-3815).
23.
MayeA.EngelA. (2012a). Time scales of sensorimotor contingencies. In ZhangH. (Ed.), Brain-Inspired Cognitive Systems 2012 (BICS 2012) (Vol. 7366, pp. 240–249). Berlin, Heidelberg: Springer.
24.
MayeA.EngelA. (2012b). Using sensorimotor contingencies for prediction and action planning. In ZiemkeT.BalkeniusC.HallamJ. (Eds.), From Animals to Animats 12 (Vol. 7426, pp. 106–116). Berlin, Heidelberg: Springer.
25.
McGannM. (2007). Enactive theorists do it on purpose: Toward an enactive account of goals and goal-directedness. Phenomenology and the Cognitive Sciences, 6(4), 463–483.
26.
ModayilJ.WhiteA.SuttonR. (2012). Multi-timescale nexting in a reinforcement learning robot. In ZiemkeT.BalkeniusC.HallamJ. (Eds.), From Animals to Animats 12 (Vol. 7426, pp. 299–309). Berlin, Heidelberg: Springer.
27.
MöllerR.SchenckW. (2008). Bootstrapping cognition from behavior – a computerized thought experiment. Cognitive Science, 32(3), 504–542.
28.
O’ReganJ.NoëA. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24, 939–1031.
29.
OudeyerP.-Y.KaplanF. (2004). Intelligent adaptive curiosity: a source of self-development. In BerthouzeL.KozimaH.PrinceC. G.SandiniG.StojanovG.MettaG.BalkeniusC. (Eds.), Proceedings of the 4th International Workshop on Epigenetic Robotics (Vol. 117, pp. 127–130). Lund University Cognitive Studies.
30.
PfeiferR.ScheierC. (1997). Sensory-Motor Coordination: The Metaphor and Beyond. Robotics and Autonomous Systems, 20, 157–178.
31.
RibesA.CerquidesJ.DemirisY.MantarasR. de. (2012). Incremental learning of an optical flow model for sensorimotor anticipation in a mobile robot. In Development and Learning and Epigenetic Robotics (ICDL), 2012 IEEE International Conference on (pp. 1–2).
32.
RusselS.NorvigP. (2003). Artificial Intelligence: A Modern Approach. Pearson Education Inc.
33.
SahinE.CakmakM.DogarM. R.UgurE.ÜcolukG. (2007). To afford or not to afford: A new formalization of affordances towards affordance based robot control. Adaptive Behavior, 15(4), 447–472.
34.
SchneiderK.SchmaltH. (1994). Motivation (2 ed.). Stuttgart: Verlag W. Kohlhammer.
35.
SeoM.LeeE.AverbeckB. B. (2012). Action selection and action value in frontal-striatal circuits. Neuron, 74(5), 947–60.
36.
SethA. K. (2007). The ecology of action selection: insights from artificial life. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1545–58.
37.
SpornsO.EdelmanG. M. (1993). Solving Bernstein’s problem: a proposal for the development of coordinated movement by selection. Child Development, 64(4), 960–981.
38.
SuttonR.BartoA. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
39.
SuttonR.ModayilJ.DelpM.DegrisT.PilarskiP.WhiteA.PrecupD. (2011). Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (pp. 761–768).
40.
UgurE.DogarM.CakmakM.SahinE. (2007). Curiosity-driven learning of traversability affordance on a mobile robot. In Development and Learning, 2007. ICDL 2007. IEEE 6th International Conference on (pp. 13–18).
41.
UgurE.OztopE.SahinE. (2009). Affordance learning from range data for multi-step planning. In Proceedings of the Ninth International Conference on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems. (Vol. 146, pp. 177–184).
42.
UgurE.OztopE.SahinE. (2011). Going beyond the perception of affordances: Learning how to actualize them through behavioral parameters. In Robotics and Automation, 2011. ICRA 2011. IEEE International Conference on (4768-4773).
43.
UgurE.SahinE.OztopE. (2009). Predicting future object states using learned affordances. In Computer and Information Sciences, 2009. ISCIS 2009. 24th International Symposium on (pp. 415–419).
