Animals develop and use cognitive maps, which are internal models of the external environment, to understand the spatial characteristics of their natural environment. Previous studies have shown that a hierarchical structure of recurrent neural networks contributes to the extraction of high-level concepts in sequential sensorimotor experiences. However, the previous studies did not focus on the spatial aspects of these experiences and did not acquire cognitive maps. We modified previous models and trained the proposed model with the visuomotor experiences of an agent in a simulated two-dimensional environment. The proposed model was trained to predict future visual and motion inputs even when only one modality was provided (crossmodal prediction). The proposed model correctly predicted visual images, even when the agent experienced unknown paths. Comparisons of the crossmodal predictions of the models under different conditions revealed that the crossmodal predictions related to motion resulted in self-organization of the cognitive map. Further experiments of mental simulation abilities showed that two-way crossmodal predictions (from vision and motion only) were required for consistent generation of vision and motion. These results indicated that predictive learning involving integrated vision and motion was necessary for self-organization of spatial recognition with a cognitive map.
AbadiM.AgarwalA.BarhamP.BrevdoE.ChenZ.CitroC.ZhengX. (2016, November). TensorFlow: Large-scale machine learning on heterogeneous distributed systems. Paper presented at the 12th USENIX Symposium on Operating System Design and Implementation, Savannah, GA.
2.
ChenG.KingJ. A.BurgessN.O’KeefeJ. (2013). How vision and movement combine in the hippocampal place code. Proceedings of the National Academy of Sciences, 110, 378–383.
3.
ChoK.van MerriënboerB.GulcehreC.BahdanauD.BougaresF.SchwenkH.BengioY. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), 25–29 October 2014, Doha, Qatar.
4.
ChoiM.TaniJ. (2016). Predictive coding for dynamic vision: Development of functional hierarchy in a multiple spatio-temporal scales RNN model. NIPS Salon des Refusés, 5–10 December, Barcelona, Spain.
5.
ChungJ.GulcehreC.ChoK. H.BengioY. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. NIPS 2014 Workshop on Deep Learning. 12 December, Montreal, Canada.
6.
ClevertD. A.UnterthinerT.HochreiterS. (2016). Fast and accurate deep network learning by exponential linear units (ELUs). International Conference on Learning Representations (ICLR 2016). 2–4 May, San Juan, Puerto Rico.
7.
DavidsonT. J.KloostermanF.WilsonM. A. (2009). Hippocampal replay of extended experience. Neuron, 63, 497–507.
8.
FyhnM.MoldenS.WitterM. P.MoserE. I.MoserM.-B. (2004). Spatial representation in the entorhinal cortex. Science, 305, 1258–1264.
9.
HeldR.HeinA. (1963). Movement-produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56, 872–876.
10.
HwangJ.JungM.KimJ.TaniJ. (2016, 19–22September). A deep learning approach for seamless integration of cognitive skills for humanoid robots. The Sixth Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Cergy-Pontoise, France.
11.
JauffretA.CuperlierN.GaussierP. (2015). From grid cells and visual place cells to multimodal place cell: A new robotic architecture. Frontiers in Neurorobotics, 9, 1–22.
12.
JungM.HwangJ.TaniJ. (2015). Self-organization of spatio-temporal hierarchy via learning of dynamic visual image patterns on action sequences. PLoS ONE, 10, e0131214.
13.
KawatoM.FurukawaK.SuzukiR. (1987). A hierarchical neural-network model for control and learning of voluntary movement. Biological Cybernetics, 57, 169–185.
14.
KingmaD. P.BaJ. L. (2015). Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR 2015), 7–9 May 2015, San Diego, CA, USA.
15.
LeCunY.BengioY.HintonG. (2015). Deep learning. Nature, 521, 436–444.
16.
McNaughtonB. L.BattagliaF. P.JensenO.MoserE. I.MoserM. B. (2006). Path integration and the neural basis of the ‘cognitive map’. Nature Reviews Neuroscience, 7, 663–678.
17.
MilfordM. J.WyethG. F.PrasserD. (2004, 26April–1May). RatSLAM: A hippocampal model for simultaneous localization and mapping. In IEEE International Conference on Robotics and Automation (ICRA 2004), New Orleans, LA.
18.
MöllerR.SchenckW. (2008). Bootstrapping cognition from behavior-a computerized thought experiment. Cognitive Science, 32, 504–542.
19.
MoserE. I.KropffE.MoserM.-B. (2008). Place cells, grid cells, and the brain’s spatial representation system. Annual Review of Neuroscience, 31, 69–89.
20.
MullerR. U.KubieJ. L. (1987). The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells. Journal of Neuroscience, 7, 1951–1968.
21.
O’KeefeJ.DostrovskyJ. (1971). The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Research, 34, 171–175.
22.
ÓlafsdóttirH. F.BarryC.SaleemA. B.HassabisD.SpiersH. J. (2015). Hippocampal place cells construct reward related sequences through unexplored space. Elife, 4, e06063.
23.
PaineR. W.TaniJ. (2005). How hierarchical control self-organizes in artificial adaptive systems. Adaptive Behavior, 13, 211–225.
24.
PhiliponaD.O’ReganJ. K.NadalJ.-P. (2003). Is there something out there? Inferring space from sensorimotor dependencies. Neural Computation, 15, 2029–2049.
25.
TaniJ. (1996). Model-based learning for mobile robot navigation from the dynamical systems perspective. IEEE Transactions on Systems, Man, and Cybernetics–Part B: Cybernetics, 26, 421–436.
26.
TaniJ.NolfiS. (1999). Learning to perceive the world as articulated: An approach for hierarchical learning in sensory-motor systems. Neural Networks, 12, 1131–1141.
27.
TaubeJ. S.MullerR. U.RanckJ. B.Jr. (1990). Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis. Journal of Neuroscience, 10, 420–435.
28.
TerekhovA. V.O’ReganJ. K. (2016). Space as an invention of active agents. Frontiers in Robotics and AI, 3, 4. DOI: 10.3389/frobt.2016.00004.
29.
TolmanE. C. (1948). Cognitive maps in rats and men. Psychological Review, 55, 189–208.
30.
Villacorta-AtienzaJ. A.CalvoC.MakarovV. A. (2015). Prediction-for-CompAction: Navigation in social environments using generalized cognitive maps. Biological Cybernetics, 109, 307–320.
31.
Villacorta-AtienzaJ. A.MakarovV. A. (2013). Neural network architecture for cognitive navigation in dynamic environments. IEEE Transactions on Neural Networks and Learning Systems, 24, 2075–2087.
32.
WilliamsR. J.ZipserD. (1995). Gradient-based learning algorithms for recurrent networks and their computational complexity. In ChauvinY.RumelhartD. E. (Eds.), Back-propagation: Theory, architectures and applications (pp. 433–486). Hillsdale, NJ: Erlbaum.
33.
WyssR.KönigP.VerschureP. F. (2006). A model of the ventral visual system based on temporal stability and local memory. PLoS Biology, 4, e120.
34.
XingjianS.ChenZ.WangH.YeungD. Y.WongW. K.WooW. C. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In CorinnaC.NeilD. L.DanielD. L.MasashiS.RomanG. (Eds.), Advances in Neural Information Processing Systems (pp. 802–810), New York, US: Curran Associates.
35.
YamashitaY.TaniJ. (2008). Emergence of functional hierarchy in a multiple timescale neural network model: A humanoid robot experiment. PLoS Computational Biology, 4, e1000220.
