Visual object-recognition plays a crucial role in animals that utilize visual information. In this study, we address the prey-predator recognition problem by optimizing artificial convolutional neural networks, based on neuroethological studies on toads. After the optimization of the overall network by supervised learning, the network showed a reasonable performance, even though various types of image noise existed. Also, we modulated the network after the optimization process based on the computational theory of classical conditioning and the reinforcement learning algorithm for the adaptation to environmental changes. This adaptation was implemented by separated modules that implement the “innate” term and “acquired” term of outputs. The modulated network exhibited behaviors that were similar to those of real toads. The neural basis of the amphibian visual information processing and the behavioral modulation mechanism have been substantially studied by biologists. Recent advances in parallel distributed processing technologies may enable us to develop fully autonomous, adaptive artificial agents with high-dimensional input spaces through end-to-end training methodology.
BadenT.BerensP.FrankeK.RosónM. R.BethgeM.EulerT. (2016). The functional diversity of retinal ganglion cells in the mouse. Nature, 529(7586), 345–350.
2.
BartoA. G. (1995). Adaptive critics and the basal ganglia. Models of information processing in the basal ganglia, 215.
3.
BartoA. G.SuttonR. S.AndersonC. W. (1983). Neuronlike adaptive elements that can solve difficult learning control problems.IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(5), 834–846.
4.
BellA. J.SejnowskiT. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37(23), 3327–3338.
BöhmerW.SpringenbergJ. T.BoedeckerJ.RiedmillerM.ObermayerK. (2015). Autonomous learning of state representations for control: An emerging field aims to autonomously learn state representations for reinforcement learning agents from their real-world sensor observations. KIK ünstliche Intelligenz, 1–10.
7.
BrzoskaJ.SchneiderH. (1978). Modification of prey-catching behavior by learning in the common toad (bufo b. bufo [l], anura, amphibia): Changes in responses to visual objects and effects of auditory stimuli. Behavioural Processes, 3(2), 125–136.
8.
Cervantes-PérezF.LaraR.ArbibM. (1985). A neural model of interactions subserving preypredator discrimination and size preference in anuran amphibia. Journal of Theoretical Biology, 113(1), 117–152.
9.
CollettT. (1982). Do toads plan routes? A study of the detour behaviour of bufo viridis. Journal of Comparative Physiology, 146(2), 261–271.
10.
DayanP.AbbottL. F. (2001). Theoretical neuroscience (Vol. 806). Cambridge, MA: MIT Press.
11.
DengJ.DongW.SocherR.LiL.-J.LiK.Fei-FeiL. (2009). Imagenet: A large-scale hierarchical image database. In Computer vision and pattern recognition, 2009. cvpr 2009. ieee conference on (pp. 248–255).
12.
DisseD.EwertJ.-P.HesebeckO.LangerF. (1992). Odann: Objekt dimensionieren mit ann (Tech. Rep.). Kassel, Germany: University of Kassel.
13.
DoyaK.UchibeE. (2005). The cyber rodent project: Exploration of adaptive mechanisms for self-preservation and self-reproduction. Adaptive Behavior, 13(2), 149–160.
14.
EikmannsK.-H. (1955). Verhaltensphysiologische untersuchungen über den beutefang und das bewegungssehen der erdkröte (bufo bufo l.). Zeitschrift für Tierpsychologie, 12(2), 229–253.
15.
EwertJ.-P. (1974). The neural basis of visually guided behavior. Scientific American, 230(3), 34–42.
16.
EwertJ.-P. (1980). Neuroethology–an introduction to the neurophysiological fundamentals of behaviour. New York: Springer-Verlag.
17.
EwertJ.-P. (1997). Neural correlates of key stimulus and releasing mechanism: a case study and two concepts. Trends in Neurosciences, 20(8), 332–339.
18.
EwertJ.-P. (2004) Motion perception shapes the visual world of amphibians. In PreteF. R. (Ed.), Complex worlds from simpler nervous systems (pp. 116–160). Cambridge, MA: MIT Press.
19.
EwertJ.-P.BurghagenH.AlbrechtL.KepperJ. (1982). Effects of background structure on the discrimination of configurational moving prey dummies by toadsbufo bufo (l.). Journal of Comparative Physiology, 147(2), 179–187.
20.
EwertJ.-P.Buxbaum-ConradiH.DreisvogtF.GlagowM.Merkel-HarffC.RöttgenA.. . . SchwippertW. (2001). Neural modulation of visuomotor functions underlying preycatching behaviour in anurans: perception, attention, motor performance, learning. Comparative Biochemistry and Physiology Part A:Molecular & Integrative Physiology, 128(3), 417–460.
21.
EwertJ.-P.Buxbaum-ConradiH.FingerlingS.Schürg-PfeifferE.BenekeT. W.DingesA. W.. . . SchwippertW. W. (1992). Adapted and adaptive properties in neural networks for visual pattern discrimination: a neurobiological analysis toward neural engineering. Adaptive Behavior, 1(2), 123–154.
22.
EwertJ.-P.DingesA.FinkenstädtT. (1994). Species-universal stimulus responses, modified through conditioning, reappear after telencephalic lesions in toads. Naturwissenschaften, 81(7), 317–320.
23.
EwertJ.-P.HockF. (1972). Movement-sensitive neurones in the toad’s retina. Experimental Brain Research, 16(1), 41–59.
24.
EwertJ.-P.HockF.von WietersheimA. (1974). Thalamus, praetectum, tectum: Retinale topographie und physiologische interaktionen bei der krötebufo bufo (l.). Journal of Comparative Physiology, 92(4), 343–356.
25.
EwertJ.-P.KehlW. (1978). Configurational prey-selection by individual experience in the toadbufo bufo. Journal of Comparative Physiology, 126(2), 105–114.
26.
FingerlingS.EwertJ.-P.MenzelR.PfeifferE. (1993). From the toad to a robot: Implementation of neurobiological principles of object discrimination in neural engineering. Naturwissenschaften, 80(7), 321–324.
27.
FinkenstädtT. (1989). Visual associative learning: searching for behaviorally relevant brain structures in toads. Visuomotor Coordination, 799–832.
28.
FukushimaK. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202.
29.
GaillardF.ArbibM. A.CorbachoF. J.LeeH. B. (1998). Modeling the physiological responses of anuran r3 ganglion cells. Vision Research, 38(17), 2551–2568.
30.
GollischT.MeisterM. (2008). Modeling convergent on and off pathways in the early visual system. Biological Cybernetics, 99(4-5), 263–278.
31.
GordonJ.HoodD. (1976). Anatomy and physiology of the frog retina. In FiteK. (Ed.), The amphibian visual system: A multidisciplinary approach (pp. 29–86). New York: Academic Press.
32.
GrüsserO.-J.Grüsser-CornehlsU.FinkelsteinD.HennV.PatutschnikM.ButenandtE. (1967). A quantitative analysis of movement detecting neurons in the frog’s retina. Pflüger’s Archiv für die gesamte Physiologie des Menschen und der Tiere, 293(1), 100–106.
33.
HubelD. H.WieselT. N. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160(1), 106.
34.
HubelD. H.WieselT. N. (1965). Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. Journal of Neurophysiology, 28(2), 229–289.
35.
IjspeertA. J. (2001). A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84(5), 331–348.
36.
IjspeertA. J.CrespiA.RyczkoD.CabelguenJ.-M. (2007). From swimming to walking with a salamander robot driven by a spinal cord model. Science, 315(5817), 1416–1420.
37.
IngleD. (1976). Spatial vision in anurans. In FiteK. (Ed.), The amphibian visual system: A multidisciplinary approach (pp. 119–140). New York: Academic Press.
JonschkowskiR.BrockO. (2014). State representation learning in robotics: Using prior knowledge about physical interaction. In Proceedings of Robotics, Science and Systems.
40.
KingmaD.BaJ. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
41.
KrizhevskyA.SutskeverI.HintonG. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 1097–1105.
LangeS.RiedmillerM.VoigtlanderA. (2012). Autonomous reinforcement learning on raw visual input data in a real world application. In Neural Networks (IJCNN), the 2012 International Joint Conference on (pp. 1–8). IEEE.
44.
LaraR.ArbibM. A. (1985). A model of the neural mechanisms responsible for pattern recognition and stimulus specific habituation in toads. Biological Cybernetics, 51(4), 223–237.
45.
LaraR.CarmonaM.DazaF.CruzA. (1984). A global model of the neural mechanisms responsible for visuomotor coordination in toads. Journal of Theoretical Biology, 110(4), 587–618.
46.
LeCunY.BottouL.BengioY.HaffnerP. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
47.
LettvinJ. Y.MaturanaH. R.McCullochW. S.PittsW. H. (1959). What the frog’s eye tells the frog’s brain. Proceedings of the IRE, 47(11), 1940–1951.
48.
LockA.CollettT. (1979). A toad’s devious approach to its prey: a study of some complex uses of depth vision. Journal of Comparative Physiology, 131(2), 179–189.
49.
LockA.CollettT. (1980). The three-dimensional world of a toad. Proceedings of the Royal Society of London B: Biological Sciences, 206(1165), 481–487.
50.
MattnerJ.LangeS.RiedmillerM. (2012). Learn to swing up and balance a real pole based on raw visual input data. In Neural Information Processing (pp. 126–133). NIPS Foundation.
51.
MnihV.KavukcuogluK.SilverD.RusuA. A.VenessJ.BellemareM. G., . . . others (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
52.
NgA. Y.RussellS. (2000). Algorithms for inverse reinforcement learning. In Proc. 17th international conference on machine learning, San Francisco, CA.
53.
PetreskaB. (2004). A neural visuomotor controller for asimulated Salamander Robot. Dissertation. cole Polytechnique Fdrale de Lausanne.
54.
RescorlaR. A.WagnerA. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non reinforcement. In BlackA. H.ProkasyW. F. (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York: Appleton-Century-Crofts.
55.
ShibataK.IidaM. (2003). Acquisition of box pushing by direct-vision-based reinforcement learning. In SICE 2003 annual conference, Fukui, Japan (Vol. 3, pp. 2322–2327).
56.
ShibataK.KawanoT. (2008). Learning of action generation from raw camera images in a realworld-like environment by simple coupling of reinforcement learning and a neural network. In KppenM.KasabovN.CoghillG. (Eds.), Advances in neuro-information processing (pp. 755–762).
57.
SuttonR. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1), 9–44.
58.
SuttonR. S.BartoA. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
59.
TeetersJ. L.ArbibM. (1991). A model of anuran retina relating interneurons to ganglion cell responses. Biological Cybernetics, 64(3), 197–207.
60.
TokuiS.OonoK.HidoS.ClaytonJ. (2015). Chainer: a next-generation open source framework for deep learning. In Workshop on machine learning systems at neural information processing systems (NIPS). Montral, Quebec: Organization: NIPS Foundation.
61.
Van der MaatenL.HintonG. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9, 2579–2605.
62.
WangD. (1993). Modeling stimulus specific habituation: The role of the primordial hippocampus. In EeckmanF. H.BowerJ. M. (Eds.), Computation and neural systems (pp. 103–107). New York: Springer.
63.
WangD.ArbibM. A. (1991). How does the toad’s visual system discriminate different worm-like stimuli?Biological Cybernetics, 64(3), 251–261.
64.
WatterM.SpringenbergJ. T.BoedeckerJ.RiedmillerM. (2015). Embed to control: A locally linear latent dynamics model for control from raw images. Advance online publication. arXiv:1506.07365.
65.
WidrowB.HoffM. E. (1960). Adaptive switching circuits. In Institute of Radio Engineers (Ed.), Western Electronic Show and Convention, Convention Record (Vol. 4, pp. 96–104). New York: Institute of Radio Engineers.
66.
WulfmeierM.OndruskaP.PosnerI. (2015). Deep inverse reinforcement learning. Advance online publication. arXiv:1507.04888.
