In this article, we present a hierarchical Bayesian framework for multimodal active perception, devised to be emergent, scalable and adaptive. This framework, while not strictly neuromimetic, finds its roots in the role of the dorsal perceptual pathway of the human brain. Its composing models build upon a common spatial configuration that is naturally fitting for the integration of readings from multiple sensors using a Bayesian approach devised in previous work. The framework presented in this article is shown to adequately model human-like active perception behaviours, namely by exhibiting the following desirable properties: high-level behaviour results from low-level interaction of simpler building blocks; seamless integration of additional inputs is allowed by the Bayesian Programming formalism; initial ‘genetic imprint’ of distribution parameters may be changed ‘on the fly’ through parameter manipulation, thus allowing for the implementation of goal-dependent behaviours (i.e. top-down influences).
AloimonosJ.WeissI.BandyopadhyayA. (1987). Active vision. International Journal of Computer Vision, 1, 333–356.
2.
BajcsyR. (1985). Active perception vs passive perception. In Third IEEE Workshop on Computer Vision (pp. 55–59), Bellaire, Michigan.
3.
BallardD. H. (1999). An introduction to natural computation. Cambridge, MA: MIT Press.
4.
BessièreP.LaugierC.SiegwartR. (Eds.). (2008). Probabilistic reasoning and decision making in sensory-motor systems (Vol. 46). Berlin: Springer.
5.
BohgJ.Barck-holstC.HuebnerK.RalphM.RasolzadehB.SongD.KragicD. (2009). Towards grasp-oriented visual perception for humanoid robots. International Journal of Humanoid Robotics, 3(3), 387–434.
6.
BreazealC.EdsingerA.FitzpatrickP.ScassellatiB. (2001). Active vision for sociable robots. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 31(5), 443–453.
7.
BuntineW. L. (1994). Operations for learning with graphical models. Journal of Artificial Intelligence Research (AI Access Foundation), 2, 159–225.
8.
CarmiR.IttiL. (2006). Causal saliency effects during natural vision. In Proceedings of the Symposium on Eye Tracking Research Applications. Vol. 1. ACM Press (pp. 11–18).
9.
ColasF.DiardJ.BessièreP. (2010). Common Bayesian models for common cognitive issues. Acta Biotheoretica, 58(2–3), 191–216.
10.
ColasF.FlacherF.TannerT.BessièreP.GirardB. (2009). Bayesian models of eye movement selection with retinotopic maps. Biological Cybernetics, 100, 203–214.
11.
CorbettaM.ShulmanG. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 201–215.
12.
CroonG. C. H. E. deSprinkhuizen-KuyperI. G.PostmaE. O. (2009). Comparing active vision models. Image and Vision Computing, 27(4), 374–384.
13.
CuttingJ. E.VishtonP. M. (1995). Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth. In: EpsteinW.RogersS. (Eds.), Handbook of perception and cognition (Vol. 5; Perception of space and motion) (pp. 69–117). San Diego: Academic Press.
14.
DankersA.BarnesN.ZelinskyA. (2005). Active vision for road scene awareness. In IEEE Intelligent Vehicles Symposium (IVS05) (pp. 187–192), Las Vegas, USA.
15.
DankersA.BarnesN.ZelinskyA. (2007). A Reactive vision system: Active-dynamic saliency. In 5th International Conference on Computer Vision Systems, Bielefeld, Germany.
16.
DoyaK.IshiiS.PougetA.RaoR. P. N. (Eds.). (2007). Bayesian brain — probabilistic approaches to neural coding. Cambridge, MA: MIT Press.
17.
ElfesA. (1989). Using occupancy grids for mobile robot perception and navigation. IEEE Computer, 22(6), 46–57.
18.
FerreiraJ. F.BessièreP.MekhnachaK.LoboJ.DiasJ.LaugierC. (2008). Bayesian models for multimodal perception of 3D structure and motion. In International Conference on Cognitive Systems (CogSys 2008) (pp. 103–108), University of Karlsruhe, Karlsruhe, Germany.
19.
FerreiraJ. F.Castelo-BrancoM. (2007). 3D structure and motion multimodal perception (State-of-the-Art Report). Institute of Systems and Robotics and Institute of Biomedical Research in Light and Image, University of Coimbra. (Bayesian Approach to Cognitive Systems (BACS) European Project)
20.
FerreiraJ. F.LoboJ.DiasJ. (2011). Bayesian real-time perception algorithms on GPU — Real-time implementation of Bayesian models for multimodal perception using CUDA. Journal of Real-Time Image Processing, 6(3), 171–186.
21.
FerreiraJ. F.PinhoC.DiasJ. (2008a). Active exploration using Bayesian models for multimodal perception. In CampilhoA.KamelM. (Eds.), Image Analysis and Recognition, Lecture Notes in Computer Science series, International Conference ICIAR 2008 (pp. 369–378). Berlin: Springer.
22.
FerreiraJ. F.PinhoC.DiasJ. (2008b). Bayesian sensor model for egocentric stereovision. In 14a Conferência Portuguesa de Reconhecimento de Padrões Coimbra (RECPAD 2008).
23.
FerreiraJ. F.PinhoC.DiasJ. (2009). Implementation and calibration of a Bayesian binaural system for 3D localisation. In 2008 IEEE International Conference on Robotics and Biomimetics (ROBIO 2008), Bangkok, Thailand.
24.
FerreiraJ. F.PradoJ.LoboJ.DiasJ. (2009). Multimodal active exploration using a Bayesian approach. In IASTED International Conference in Robotics and Applications, (pp. 319–326), Cambridge MA, USA.
IttiL.KochC.NieburE. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.
28.
KnillD. C.PougetA. (2004). The Bayesian brain: the role of uncertainty in neural coding and computation. TRENDS in Neurosciences, 27(12), 712–719.
29.
KoeneA.MorénJ.TrifaV.ChengG. (2007). Gaze shift reflex in a humanoid active vision system. In 5th International Conference on Computer Vision Systems, Bielefeld, Germany.
30.
KoppL.GärdenforsP. (2002). Attention as a minimal criterion of intentionality in robots. Cognitive Science Quarterly, 2, 302–319.
31.
LebeltelO. (1999). Programmation Bayésienne des Robots. Unpublished doctoral dissertation, Institut National Polytechnique de Grenoble, Grenoble, France.
32.
LeeM. D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55(1), 1–7. (Special Issue on Hierarchical Bayesian Models)
33.
LuY.-C.ChristensenH.CookeM. (2007). Active binaural distance estimation for dynamic sources. In Interspeech 2007 (pp. 574–577), Antwerp, Belgium.
34.
MarrD. (1982). Vision: A computational investigation into the human representation and processing of visual information. S. Francisco: W. H. Freeman and Company.
35.
NieburE.IttiL.KochC. (1995). Modeling the “where” visual pathway. In SejnowskiT. J. (Ed.), 2nd Joint Symposium on Neural Computation, Caltech-UCSD (Vol. 5, pp. 26–35), La Jolla.
36.
NVIDIA. (2007). CUDA Programming Guide version 1.2.
37.
PearlJ. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference (Revised Second Printing ed.; MorganM. B., Ed.). Morgan Kaufmann Publishers, Inc. (Elsevier).
38.
PinhoC.FerreiraJ. F.BessièreP.DiasJ. (2008, April). A Bayesian binaural system for 3D sound-source localisation. In International Conference on Cognitive Systems (CogSys 2008) (p. 109–114), University of Karlsruhe, Karlsruhe, Germany.
39.
PougetA.DayanP.ZemelR. (2000). Information processing with population codes. Nature Reviews Neuroscience, 1, 125–132. (Review)
40.
ShibataT.VijayakumarS.ConradtJ.SchaalS. (2001). Biomimetic oculomotor control. Adaptive Behaviour - Special Issue on Biologically Inspired and Biomimetic System, 9(3–4), 189–208.
41.
ShiffrinR.LeeM.WagenmakersE.-J.KimW. J. (2008). A survey of model evaluation approaches with a focus on hierarchical Bayesian methods. Cognitive Science, 32(8), 1248–1284.
42.
SimoncelliE. P. (1999). Bayesian multi-scale differential optical flow. In JähneB.HausseckerH.GeisslerP. (Eds.), Handbook of Computer Vision and Applications. Academic Press.
43.
TayC.MekhnachaK.ChenC.YguelM.LaugierC. (2008). An efficient formulation of the Bayesian occupation filter for target tracking in dynamic environments. International Journal of Autonomous Vehicles, 6(1–2), 155–171.
44.
TsotsosJ.ShubinaK. (2007). Attention and visual search: Active robotic vision systems that search. In The 5th International Conference on Computer Vision Systems, March 21–24, Bielefeld.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
