In this paper we introduce a robust descriptor for matching vertical lines among two or more images from an omnidirectional camera. Furthermore, in order to make such a descriptor usable in the framework of indoor mobile robotics, this paper introduces a new simple strategy to extrinsically self-calibrate the omnidirectional sensor with the odometry reference system. In the first part of this paper we describe how to build the feature descriptor. We show that the descriptor is very distinctive and is invariant to rotation and slight changes in illumination. The robustness of the descriptor is validated through real experiments on a wheeled robot. The second part of the paper is devoted to the extrinsic self-calibration of the camera with the odometry reference system. We show that by implementing an extended Kalman filter that fuses the information from the visual features with the odometry, it is possible to extrinsically and automatically calibrate the camera while the robot is moving. In particular, it is theoretically shown that only one feature suffices to perform the calibration. Experimental results validate the theoretical contributions.
Ayache, N. (1990) Stereovision and Sensor Fusion. Cam-bridge, MA, The MIT Press.
2.
Ayache, N. and Faugeras, O. (1987) Building a consistent 3D representation of a mobile robot environment by combining multiple stereo views. Proceedings of IJCAI, pp. 808- 810.
3.
Baillard, C., Schmid, C., Zisserman, A. and Fitzgibbon, A. (1999) Automatic line matching and 3D reconstruction of buildings from multiple views. Proceedings of the SPRS Conference on Automatic Extraction of GIS Objects from Digital Imagery, (International Archives of Photogrammetry and Remote Sensing, Vol. 32, Part 3-2W5). International Society for Photogrammetry and Remote Sensing, pp. 69- 80.
4.
Bar-Shalom, Y. and Fortmann, T.E. (1988) Tracking and Data Association (Mathematics in Science and Engineering, Vol. 179). New York, Academic Press.
5.
Baumberg, A. (2000) Reliable feature matching across widely separated views. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head, SC, pp. 774-781.
6.
Brassart, E., Delahoche, L., Cauchois, C., Drocourt, C., Pegard, C. and Mouaddib, E.M. (2000) Experimental results got with the omnidirectional vision sensor: SYCLOP. Proceedings of the International Workshop on Omnidirectional Vision (OMNIVIS 2000).
7.
Briggs, A., Li, Y., Scharstein, D. and Wilder, M. (2006) Robot navigation using 1D panoramic images. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2006), Orlando, FL, May 2006.
8.
Chong, K.S. and Kleeman, L. (1997) Accurate odometry and error modelling for a mobile robot. Proceedings of the International Conference on Robotics and Automation, pp. 2783-2788.
9.
Crowley, J. and Stelmazyk, P. (1990) Measurement and integration of 3D structures by tracking edge lines. Proceedings of ECCV, pp. 269-280.
10.
Deriche, R. and Faugeras, O. (1990) Tracking line segments. Proceedings of ECCV, pp. 259-267.
11.
Goedeme, T., Tuytelaars, T. and Van Gool, L. (2004) Fast wide baseline matching with constrained camera position. Proceedings of the Conference on Computer Vision and Pattern Recognition, Washington, DC, pp. 24-29.
12.
Gonzalez, R. and Woods, R. (2002) Digital Image Processing, 2nd edition. Reading, MA, Addison-Wesley/ Englewood Cliffs, NJ, Prentice-Hall.
13.
Gros, P. (1995) Matching and clustering: two steps towards object modelling in computer vision. The International Journal of Robotics Research, 14(6): 633-642.
14.
Hermann, R. and Krener, A.J. (1977) Nonlinear controllability and observability. IEEE Transactions on Automatic Control, 22(5): 728-740.
15.
Hesch, J.A., Mourikis, A.I. and Roumeliotis, S.I. (2008) Determining the camera to robot-body transformation from planar mirror reflections. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2008), Nice, France, September 2008.
16.
Horaud, R. and Skordas, T. (1989) Stereo correspondence through feature grouping and maximal cliques. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(11): 1168-1180.
17.
Huttenlocher, D.P., Klanderman, G.A. and Rucklidge, W.J. (1993) Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9): 850-863.
18.
Isidori, A. (1995) Nonlinear Control Systems, 3rd edition. Berlin, Springer.
19.
Kadir, T., Zisserman, A. and Brady, M. (2004) An affine invariant salient region detector. Proceedings of the 7th European Conference on Computer Vision, Prague, Czech Republic, Vol. I, pp. 228-241.
20.
Lindeberg, T. (1998) Feature detection with automatic scale selection . International Journal of Computer Vision, 30(2): 79-116.
21.
Lowe, D.G. (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2): 91-110.
22.
Martinelli, A., Scaramuzza, D. and Siegwart, R. (2006) Automatic self-calibration of a vision system during robot motion. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2006).
23.
Martinelli, A., Tomatis, N. and Siegwart, R. (2007) Simultaneous localization and odometry self calibration for mobile robot. Autonomous Robots, 22: 75-85.
24.
Matas, J., Chum, O., Urban, M. and Pajdla, T. (2002) Robust wide baseline stereo from maximally stable extremal regions. Proceedings of the 13th British Machine Vision Conference, Cardiff, UK, pp. 384-393.
25.
Mauthner, T., Fraundorfer, F. and Bischof, H. (2006) Region matching for omnidirectional images using virtual camera planes. Proceedings of the Computer Vision Winter Workshop 2006, Telc, Czech Republic.
26.
Medioni, G. and Nevatia, R. (1985) Segment-based stereo matching. Computer Vision , Graphics and Image Processing, 31: 2-18.
27.
Micusik, B. and Pajdla, T. (2006) Structure from motion with wide circular field of view cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7): 1135- 1149.
28.
Mikolajczyk, K. and Schmid, C. (2001) Indexing based on scale invariant interest points . Proceedings of the 8th International Conference on Computer Vision, Vancouver, Canada, pp. 525-531.
29.
Mikolajczyk, K. and Schmid, C. (2002) An affine invariant interest point detector. Proceedings of 7th European Conference on Computer Vision, Copenhagen, Denmark.
30.
Mirzaei, F.M. and Roumeliotis, S.I. (2007) A Kalman filter-based algorithm for IMU-camera callibration. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), San Diego, CA, October 2007, pp. 2427-2434.
31.
Nayar, S.K. (1997) Catadioptric omnidirectional camera. CVPR '97: Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97). Washington, DC, IEEE Computer Society, p. 482.
32.
Prasser, D., Wyeth, G. and Milford, M.J. (2004) Experiments in outdoor operation of RatSLAM. Proceedings of the Australian Conference on Robotics and Automation , Canberra, Australia.
33.
Rahman, M.M., Bhattacharya, P. and Desai, B.C. (2005) Similarity searching in image retrieval with statistical distance measures and supervised learning. Pattern Recognition and Data Mining (Lecture Notes in Computer Science, Vol. 3686). Berlin, Springer, pp. 315-324.
34.
Rubner, Y., Tomasi, C. and Guibas, L.J. (2000) The earth mover's distance as a metric for image retrieval. International Journal of Computer Vision, 40(2): 99-121.
35.
Rubner, Y. et al. (2001) Empirical evaluation of dissimilarity measures for color and texture. Computer Vision and Image Understanding , 84(1): 25-43.
36.
Sagues, C., Murillo, A.C., Guerrero, J.J., Goedeme, T., Tuytelaars, T. and Van Gool, L. (2006) Hierarchical localization by matching vertical lines in omnidirectional images. Robotics and Autonomous Systems , 55(5): 372-382.
37.
Scaramuzza, D., Criblez, N., Martinelli, A. and Siegwart, R. (2007a) Robust feature extraction and matching for omnidirectional images. Proceedings of the 6th International Conference on Field and Service Robotics (FSR 2007), Chamonix, France, July 2007.
38.
Scaramuzza, D., Harati, A. and Siegwart, R. (2007b) Extrinsic self calibration of a camera and a 3d laser range finder from natural scenes. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), San Diego, CA, October 2007.
39.
Tell, D. and Carlsson, S. (2000) Wide baseline point matching using affine invariants computed from intensity profiles. Proceedings of the European Conference on Computer Vision, Dublin, Ireland, pp. 814-828.
40.
Tuytelaars, T. and Van Gool, L. (2004) Matching widely separated views based on affine invariant regions. International Journal of Computer Vision , 1(59): 61-85.
41.
Venkateswar, V. and Chellappa, R. (1995) Hierarchical stereo and motion correspondence using feature groupings. International Journal of Computer Vision , 15(3): 245-269.
42.
Yagi, Y. and Yachida, M. (1991) Real-time generation of environmental map and obstacle avoidance using omnidirectional image sensor with conic mirror. Proceedings of CVPR'91, pp. 160-165.
43.
Zhang, Z. (1994) Token tracking in a cluttered scene. Image and Vision Computing, 12(2): 110-120.
44.
Zhang, Q. and Pless, R. (2004) Constraints for heterogeneous sensor auto-calibration . Proceedings of the IEEE Workshop on Realtime 3D Sensors and Their Use, pp. 38-43.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
