The ability to represent knowledge about space and its position therein is crucial for a mobile robot. To this end, topological and semantic descriptions are gaining popularity for augmenting purely metric space representations. In this paper we present a multi-modal place classification system that allows a mobile robot to identify places and recognize semantic categories in an indoor environment. The system effectively utilizes information from different robotic sensors by fusing multiple visual cues and laser range data. This is achieved using a high-level cue integration scheme based on a Support Vector Machine (SVM) that learns how to optimally combine and weight each cue. Our multi-modal place classification approach can be used to obtain a real-time semantic space labeling system which integrates information over time and space. We perform an extensive experimental evaluation of the method for two different platforms and environments, on a realistic off-line database and in a live experiment on an autonomous robot. The results clearly demonstrate the effectiveness of our cue integration scheme and its value for robust place classification under varying conditions.
Aloimonos, J. and Shulman, D. ( 1989). Integration of Visual Modules: an Extension of the Marr Paradigm. New York, Academic Press .
2.
Althaus, P. and Christensen, H.I. ( 2003). Behaviour coordination in structured environments. Advanced Robotics, 17(7): 657-674.
3.
Andreasson, H., Treptow, A. and Duckett, T. ( 2005). Localization for mobile robots using panoramic vision, local features and particle filter. Proceedings of the International Conference on Robotics and Automation (ICRA), Barcelona, Spain.
4.
Barla, A., Odone, F. and Verri, A. ( 2003). Histogram intersection kernel for image classification . Proceedings of the International Conference on Image Processing (ICIP), Barcelona, Spain.
5.
Bay, H., Tuytelaars, T. and Van Gool, L.J. ( 2006). Surf: Speeded up robust features. Proceedings of the 9th European Conference on Computer Vision (ECCV), Graz, Austria.
6.
Blaer, P. and Allen, P. ( 2002). Topological mobile robot localization using fast vision techniques. Proceedings of the International Conference on Robotics and Automation (ICRA), Washington, DC.
7.
Bradley, D.M., Patel, R., Vandapel, N. and Thayer, S.M. ( 2005). Real-time image-based topological localization in large outdoor environments. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), August, Edmonton, AB, Canada.
8.
Buschka, P. and Saffiotti, A. ( 2002). A virtual sensor for room detection. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Lausanne, Switzerland.
9.
Caputo, B. and Dorko, G. ( 2002). How to combine color and shape information for 3D object recognition: Kernels do the trick. Neural Information Processing Systems, Vancouver, BC, Canada.
10.
Chapelle, O., Haffner, P. and Vapnik, V. ( 1999). Support vector machines for histogram-based image classification . Transactions on Neural Networks, 10(5): 1055-1064.
11.
Clark, J. and Yuille, A. ( 1990). Data Fusion for Sensory Information Processing Systems . Dordrecht, Kluwer.
12.
Cristianini, N. and Shawe-Taylor, J. ( 2000). An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge, Cambridge University Press.
13.
Demšar, J. ( 2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7: 1-30.
14.
Douillard, B., Fox, D. and Ramos, F. ( 2007). A spatiotemporal probabilistic model for multi-sensor object recognition. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), San Diego, CA.
15.
Duda, R., Hart, P. and Stork, D. ( 2001). Pattern Classification, 2nd edn. New York, Wiley.
16.
Filliat, D. ( 2007). A visual bag of words method for interactive qualitative localization and mapping. Proceedings of the International Conference on Robotics and Automation (ICRA), Rome, Italy.
17.
Fraundorfer, F., Engels, C. and Nistér, D. ( 2007). Topological mapping, localization and navigation using image collections. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), October, San Diego, CA.
18.
Freund, Y. and Schapire, R.E. ( 1995). A decision-theoretic generalization of on-line learning and an application to boosting. Proceedings of the European Conference on Computational Learning Theory, Barcelona, Spain.
19.
Galindo, C., Saffiotti, A., Coradeschi, S., Buschka, P., Fernández-Madrigal, J. and González, J. ( 2005). Multi-hierarchical semantic maps for mobile robotics . Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Edmonton, AB, Canada.
20.
Gaspar, J., Winters, N. and Santos-Victor, J. ( 2000). Vision-based navigation and environmental representations with an omni-directional camera. Transactions on Robotics and Automation, 16(6): 890-898.
21.
Koenig, S. and Simmons, R.G. ( 1998). Xavier: A robot navigation architecture based on partially observable Markov decision process models. Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, Kortenkamp, D., Bonasso, R. and Murphy, R. (eds). Cambridge, MA, MIT Press, pp. 91-122.
22.
Kuipers, B. ( 2006). An intellectual history of the spatial semantic hierarchy. Robot and Cognitive Approaches to Spatial Mapping. Berlin , Springer.
23.
Kuipers, B. and Beeson, P. ( 2002). Bootstrap learning for place recognition. Proceedings of the 18th National Conference on Artificial Intelligence (AAAI), Edmonton, AB, Canada.
24.
Linde, O. and Lindeberg, T. ( 2004). Object recognition using composed receptive field histograms of higher dimensionality. Proceedings of the International Conference on Pattern Recognition (ICPR), Cambridge, UK .
25.
Lowe, D.G. ( 2004). Distinctive image features from scale-invariant keypoints . International Journal of Computer Vision, 60(2): 91-110.
26.
Luo, J., Pronobis, A., Caputo, B. and Jensfelt, P. ( 2006). The IDOL2 Database. Technical Report, KTH, CAS/CVAP. Available at http://www.cas.kth.se/IDOL/.
27.
Luo, J., Pronobis, A., Caputo, B. and Jensfelt, P. ( 2007). Incremental learning for place recognition in dynamic environments. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), San Diego, CA.
28.
Matas, J., Marik, R. and Kittler, J. ( 1995). On representation and matching of multi-coloured objects . Proceedings of the International Conference on Computer Vision (ICCV), Boston, MA.
29.
Menegatti, E., Zoccarato, M., Pagello, E. and Ishiguro, H. ( 2004). Image-based Monte-Carlo localisation with omnidirectional images. Robotics and Autonomous Systems, 48(1): 17-30.
30.
Mozos, O.M., Stachniss, C. and Burgard, W. ( 2005). Supervised learning of places from range data using AdaBoost . Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Barcelona, Spain, pp. 1742- 1747.
31.
Mozos, O.M., Triebel, R., Jensfelt, P., Rottmann, A. and Burgard, W. ( 2007). Supervised semantic labeling of places using information extracted from sensor data. Robotics and Autonomous Systems , 55(5): 391-402.
32.
Murillo, A.C., Guerrero, J.J. and Sagues, C. ( 2007). Surf features for efficient robot localization with omnidirectional images. Proceedings of the International Conference on Robotics and Automation (ICRA), Rome, Italy.
33.
Nilsback, M.E. and Caputo, B. ( 2004). Cue integration through discriminative accumulation. Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Washington, DC.
34.
Orabona, F., Castellini, C., Caputo, B., Luo, J. and Sandini, G. ( 2007). Indoor place recognition using online independent support vector machines. Proceedings of the British Machine Vision Conference (BMVC), Warwick, UK.
35.
Platt, J.C. ( 1999). Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Learning , Schölkopf, B., Burges, C. and Smola, A. (eds). Cambridge, MA, MIT Press, pp. 185-208.
36.
Poggio, T., Torre, V. and Koch, C. ( 1985). Computational vision and regularization theory. Nature317: 314-319.
37.
Polikar, R. ( 2006). Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, 6: 21-45.
38.
Posner, I., Schroeter, D. and Newman, P.M. ( 2007). Describing composite urban workspaces. Proceedings of the International Conference on Robotics and Automation (ICRA), Rome, Italy.
39.
Pronobis, A. and Caputo, B. ( 2007). Confidence-based cue integration for visual place recognition . Proceedings of the International Conference on Intelligent Robots and Systems (IROS), San Diego, CA.
40.
Pronobis, A., Caputo, B., Jensfelt, P. and Christensen, H.I. ( 2006). A discriminative approach to robust visual place recognition . Proceedings of the International Conference on Intelligent Robots and Systems (IROS), October, Beijing , China.
41.
Pronobis, A., Mozos, O.M. and Caputo, B. ( 2008). SVMbased discriminative accumulation scheme for place recognition. Proceedings of the International Conference on Robotics and Automation (ICRA), May, Pasadena, CA.
42.
Rothganger, F., Lazebnik, S., Schmid, C. and Ponce, J. ( 2006). 3D object modeling and recognition using local affine-invariant image descriptors and multi-view spatial constraints. International Journal of Computer Vision, 66(3): 231-259.
43.
Rottmann, A., Mozos, O.M., Stachniss, C. and Burgard, W. ( 2005). Semantic place classification of indoor environments with mobile robots using boosting. Proceedings of the 20th National Conference on Artificial Intelligence (AAAI), Pittsburgh, PA.
44.
Se, S., Lowe, D.G. and Little, J. ( 2001). Vision-based mobile robot localization and mapping using scale-invariant features. Proceedings of the International Conference on Robotics and Automation (ICRA), Seoul, Korea.
45.
Siagian, C. and Itti, L. ( 2007). Biologically-inspired robotics vision monte-carlo localization in the outdoor environment. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), October, San Diego, CA.
46.
Stachniss, C., Mozos, O.M. and Burgard, W. ( 2006). Speeding-up multi-robot exploration by considering semantic place information. Proceedings of the International Conference on Robotics and Automation (ICRA), Orlando, FL.
47.
Stachniss, C., Grisetti, G., Mozos, O. and Burgard, W. ( 2007). Efficiently learning metric and topological maps with autonomous service robots. Information Technology, 49: 232- 237.
48.
Tamimi, H. and Zell, A. ( 2004). Vision based localization of mobile robots using kernel approaches. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan.
49.
Tapus, A. and Siegwart, R. ( 2005). Incremental robot mapping with fingerprints of places . Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Edmonton, AB, Canada.
50.
Tommasi, T., Orabona, F. and Caputo, B. ( 2008). Discriminative cue integration for medical image annotation. Pattern Recognition Letters, Special Issue on IMageCLEF Med Benchmark Evaluation, 29(15): 1996-2002.
51.
Topp, E.A. and Christensen, H.I. ( 2006). Topological modelling for human augmented mapping. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), October, Beijing, China.
52.
Torralba, A. ( 2003). Contextual priming for object detection. International Journal of Computer Vision, 53(2): 169-191.
53.
Torralba, A. and Sinha, P. ( 2001). Recognizing Indoor Scenes. Technical Report 2001-015, AI Memo.
54.
Torralba, A., Murphy, K.P., Freeman, W.T. and Rubin, M.A. ( 2003). Context-based vision system for place and object recognition . Proceedings of the International Conference on Computer Vision (ICCV), Nice, France.
55.
Triesch, J. and Eckes, C. ( 1998). Object recognition with multiple feature types. Proceedings of the International Conference on Artificial Neural Networks (ICANN), Skövde, Sweden.
56.
Ulrich, I. and Nourbakhsh, I. ( 2000). Appearance-based place recognition for topological localization . Proceedings of the International Conference on Robotics and Automation (ICRA), San Francisco, CA.
57.
Valgren, C. and Lilienthal, A.J. ( 2008). Incremental spectral clustering and seasons: Appearance-based localization in outdoor environments. Proceedings of the International Conference on Robotics and Automation (ICRA), Pasadena, CA.
58.
Wallraven, C., Caputo, B. and Graf, A. ( 2003). Recognition with local features: the kernel recipe. Proceedings of the International Conference on Computer Vision (ICCV), Nice, France.
59.
Weiss, C., Tamimi, H., Masselli, A. and Zell, A. ( 2007). A hybrid approach for vision-based outdoor robot localization using global and local image features. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), October, San Diego, CA.
60.
Yamauchi, B. ( 1997). A frontier-based approach for autonomous exploration . Proceedings of the International Symposium on Computational Intelligence in Robotics and Automation, Monterey, CA.
61.
Zender, H., Jensfelt, P., Mozos, O.M., Kruijff, G.-J.M. and Burgard, W. ( 2007). An integrated robotic system for spatial understanding and situated interaction in indoor environments. Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI) , Vancouver, BC, Canada.
62.
Zender, H., Mozos, O.M., Jensfelt, P., Kruijff, G.-J.M. and Burgard, W. ( 2008). Conceptual spatial representations for indoor mobile robots. Robotics and Autonomous Systems, 56(6): 493-502.
63.
Zivkovic, Z., Bakker, B. and Kröse, B. ( 2005). Hierarchical map building using visual landmarks and geometric constraints. Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Edmonton, AB, Canada.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
