The mechanisms of how the brain orchestrates multi-limb joint action have yet to be elucidated and few computational sensorimotor (SM) learning approaches have dealt with the problem of acquiring bimanual affordances. We propose a series of bidirectional (forward/inverse) SM maps and its associated learning processes that generalize from uni- to bimanual interaction (and affordances) naturally, reinforcing the motor equivalence property. The SM maps range from a SM nature to a solely sensory one: full body control, delta SM control (through small action changes), delta sensory co-variation (how body-related perceptual cues covariate with object-related ones). We make several contributions on how these SM maps are learned: (1) Context and Behavior-Based Babbling: generalizing goal babbling to the interleaving of absolute and local goals including guidance of reflexive behaviors; (2) Event-Based Learning: learning steps are driven by visual, haptic events; and (3) Affordance Gradients: the vectorial field gradients in which an object can be manipulated. Our modeling of bimanual affordances is in line with current robotic research in forward visuomotor mappings and visual servoing, enforces the motor equivalence property, and is also consistent with neurophysiological findings like the multiplicative encoding scheme.
AndrychowiczM.WolskiF.RayA.SchneiderJ.FongR.WelinderP.. . . ZarembaW. (2017). Hindsight experience replay. In GuyonI.LuxburgU.V.BengioS.WallachH.FergusR.VishwanathanS.GarnettR. (Eds.), Advances in neural information processing systems (pp. 5048–5058). Long Beach, California, US: Neural Information Processing Systems Foundation, Inc.
2.
BaranesA.OudeyerP. Y. (2010). Intrinsically motivated goal exploration for active motor learning in robots: a case study. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2010), Taipei, Taiwan. Retrieved from https://ieeexplore.ieee.org/document/5651385
3.
BaranesA.OudeyerP. Y. (2012). Active learning of inverse models with intrinsically motivated goal exploration in robots. Robotics and Autonomous Systems, 61, 49–73.
4.
BartoA.SinghS.ChenatezN. (2004). Intrinsically motivated learning of hierarchical collections of skills. In Proceedings 3rd International Conference Dvp. Learn (pp. 112–119). Cambridge, MA: MIT Press.
5.
BattagliaP. W.HamrickJ. B.TenenbaumJ. B. (2013). Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110, 18327–18332.
6.
BenureauF.FudalP.OudeyerP. Y. (2014). Reusing motor commands to learn object interaction. In Development and Learning and Epigenetic Robotics (ICDL-Epirob), 2014 Joint IEEE International Conferences on IEEE. Retrieved from https://hal.inria.fr/hal-01074822/document
7.
BerlyneD. E. (1954). A theory of human curiosity. British Journal of Psychology, 45, 180–191.
8.
BraudR.PittiA.GaussierP. (2018). A modular dynamic sensorimotor model for affordances learning, sequences planning, and tool-use. IEEE Transactions on Cognitive and Developmental Systems, 10, 72–87.
9.
BullockD.GrossbergS.GuentherF. H. (1993). A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. Journal of Cognitive Neuroscience, 5, 408–435.
10.
CohnD. A.GhahramaniZ.JordanM. I. (1996). Active learning with statistical models. Journal of Artificial Intelligence Research, 4, 129–145.
11.
CorbettaD.ThelenE. (1996). The developmental origins of bimanual coordination: A dynamic perspective. Journal of Experimental Psychology: Human Perception and Performance, 22(2), 502–522.
12.
DeciE.RyanR. M. (1985). Intrinsic Motivation and self-determination in human behavior. New York, NY: Plenum Press.
13.
EllisR.TuckerM. (2000). Micro-affordance: The potentiation of components of action by seen objects. British Journal of Psychology, 91, 451–471.
14.
Escobar-JuárezE.SchillaciG.Hermosillo-ValadezJ.Lara-GuzmánB. (2016). A self-organized internal models architecture for coding sensory–motor schemes. Frontiers in Robotics and AI, 3, Article 22.
15.
ForestierS.OudeyerP. Y. (2016a). Curiosity-driven development of tool use precursors: A computational model. In PapafragouA.GrodnerD.MirmanD.TrueswellJ. C. (Eds.), 38th Annual Conference of the Cognitive Science Society (CogSci 2016), Philadelphie, PA, United States, August 2016 (pp.1859–1864).
16.
ForestierS.OudeyerP. Y. (2016b). Modular active curiosity-driven discovery of tool use. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on IEEE (pp. 3965–3972). Daejeon, South Korea: IEEE.
17.
GibsonJ. (1986). The ecological approach to visual perception. Mahwah, NJ: Lawrence Erlbaum.
18.
HoffB.ArbibM. A. (1993). Models of trajectory formation and temporal interaction of reach and grasp. Journal of Motor Behavior, 25, 175–192.
19.
HosodaK.AsadaM. (1994). Versatile visual servoing without knowledge of true jacobian. In Intelligent Robots and Systems’ 94. “Advanced Robotic Systems and the Real World,” IROS’94. Proceedings of the IEEE/RSJ/GI International Conference on IEEE (Vol. 1, pp. 186–193). Munich, Germany: IEEE.
20.
KawaiY.ParkJ.HoriiT.OshimaY.TanakaK.MoriH.. . . AsadaM. (2013). Throwing skill optimization through synchronization and desynchronization of degree of freedom. In ChenX.StoneP.SucarL. E.van der ZantT. (Eds.), RoboCup 2012: Robot Soccer World Cup XVI (pp. 178–189). Berlin, Germany: Springer.
21.
KempC. C.EdsingerA. (2006). What can I control?: The development of visual categories for a robots body and the world that it influences. In IEEE International Conference on Development and Learning (ICDL), Special Session on Autonomous Mental Development. Bloomington, IN, USA: IEEE.
22.
KulkarniT. D.NarasimhanK.SaeediA.TenenbaumJ. (2016). Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. In Advances in neural information processing systems (pp. 3675–3683). Barcelona, Spain: Neural Information Processing Systems Foundation, Inc.
23.
LakeB. M.UllmanT. D.TenenbaumJ. B.GershmanS. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, 1–72.
24.
LoweD. G. (1999). Object recognition from local scale-invariant features. In Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on IEEE (Vol. 2, pp. 1150–1157). Kerkyra, Greece: IEEE.
25.
MaffeiG.HerrerosI.Sanchez-FiblaM.FristonK. J.VerschureP. F. (2017). The perceptual shaping of anticipatory actions. Proceedings of the Royal Society B, 284(1869). doi:10.1098/rspb.2017.1780
26.
MannellaF.SantucciV. G.SomogyiE.JacqueyL.O’ReganK.BaldassarreG. (2018). Know your body through intrinsic goals. Frontiers in Neurorobotics12, Article 30.
27.
Moulin-FrierC.OudeyerP. Y. (2013). Exploration strategies in developmental robotics: A unified probabilistic framework. In Development and Learning and Epigenetic Robotics (ICDL), 2013 IEEE Third Joint International Conference on IEEE (pp. 1–6). Osaka, Japan: IEEE.
28.
Moulin-FrierC.RouanetP.OudeyerP. Y. (2014). Explauto: An open-source Python library to study autonomous exploration in developmental robotics. In ICDL-Epirob-International Conference on Development and Learning, Epirob. Genoa, Italy: IEEE.
29.
Moulin-FrierC.Sanchez-FiblaM.VerschureP. F. (2015). Autonomous development of turn-taking behaviors in agent populations: A computational study. In International Conference on Development and Learning, ICDL/Epirob, Providence, RI. Providence, USA: IEEE.
30.
O’ReganJ. K.NoëA. (2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24, 939–973.
31.
OudeyerP. Y.KaplanF.HafnerV. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11, 265–286.
32.
PuigboJ. Y.Moulin-FrierC.VouloutsiV.Sanchez-FiblaM.HerrerosI.VerschureP. F. (2015). Skill refinement through cerebellar learning and human haptic feedback: An icub learning to paint experiment. In: Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on IEEE (pp. 447–452). Seoul, South Korea: IEEE.
33.
RolfM.AsadaM. (2014). Autonomous development of goals: From generic rewards to goal and self detection. In Development and Learning and Epigenetic Robotics (ICDL-Epirob), 2014 Joint IEEE International Conferences on IEEE (pp. 187–194). Genoa, Italy: IEEE.
34.
RolfM.SteilJ. J.GiengerM. (2010). Goal babbling permits direct learning of inverse kinematics. IEEE Transactions on Autonomous Mental Development, 2, 216–229.
35.
Sánchez-FiblaM.DuffA.VerschureP. F. (2011). The acquisition of intentionally indexed and object centered affordance gradients: A biomimetic controller and mobile robotics benchmark. In Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on IEEE (pp. 1115–1121), San Francisco.
36.
Sanchez-FiblaM.DuffA.VerschureP. F. (2013). A sensorimotor account of visual and tactile integration for object categorization and grasping. In Robotics and Automation (ICRA), 2013 IEEE International Conference on IEEE (pp. 107–112). Karlsruhe, Germany: IEEE.
37.
Sánchez-FiblaM.Moulin-FrierC.ArsiwallaX.VerschureP. (2017). A correlational analysis of multiagent sensorimotor interactions: Clustering autonomous and controllable entities. Retrieved from https://arxiv.org/abs/1711.08333
38.
SchmidhuberJ. (1991). A possibility for implementing curiosity and boredom in model-building neural controllers. In MeyerJ. A.WilsonS. W. (Eds.), Proceedings of SAB’91 (pp. 222–227). Paris, France: MIT Press.
39.
SchmidhuberJ. (2010). Formal theory of creativity, fun, and intrinsic motivation (1990–2010). IEEE Transactions on Autonomous Mental Development, 2, 230–247.
40.
SimionF.RegolinL.BulfH. (2008). A predisposition for biological motion in the newborn baby. Proceedings of the National Academy of Sciences, 105, 809–813.
41.
SpornsO.EdelmanG. M. (1993). Solving Bernstein’s problem: A proposal for the development of coordinated movement by selection. Child Development, 64, 960–981.
42.
TakiyamaK.SakaiY. (2016). Balanced motor primitive can explain generalization of motor learning effects between unimanual and bimanual movements. Scientific Reports, 6, Article 23331.
43.
ThillS.CaligioreD.BorghiA. M.ZiemkeT.BaldassarreG. (2013). Theories and computational models of affordance and mirror systems: An integrative review. Neuroscience & Biobehavioral Reviews, 37, 491–521.
UgurE.NagaiY.SahinE.OztopE. (2015). Staged development of robot skills: Behavior formation, affordance learning and imitation with motionese. IEEE Transactions on Autonomous Mental Development, 7, 119–139. doi:10.1109/TAMD.2015.2426192
46.
VerschureP. F.PennartzC. M.PezzuloG. (2014). The why, what, where, when and how of goal-directed choice: Neuronal and computational principles. Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1655), 20130483.
47.
VerschureP. F.VoegtlinT.DouglasR. J. (2003). Environmentally mediated synergy between perception and behaviour in mobile robots. Nature, 425, 620–624.
48.
YokoiA.HirashimaM.NozakiD. (2011). Gain field encoding of the kinematics of both arms in the internal model enables flexible bimanual action. Journal of Neuroscience, 31, 17058–17068.
49.
YokoiA.HirashimaM.NozakiD. (2014). Lateralized sensitivity of motor memories to the kinematics of the opposite arm reveals functional specialization during bimanual actions. Journal of Neuroscience, 34, 9141–9151.
50.
ZechP.HallerS.LakaniS. R.RidgeB.UgurE.PiaterJ. (2017). Computational models of affordance in robotics: A taxonomy and systematic classification. Adaptive Behavior, 25, 235–271.
