Wide-angle images exhibit significant distortion for which existing scale-space detectors such as the scale-invariant feature transform (SIFT) are inappropriate. The required scale-space images for feature detection are correctly obtained through the convolution of the image, mapped to the sphere, with the spherical Gaussian. A new visual key-point detector, based on this principle, is developed and several computational approaches to the convolution are investigated in both the spatial and frequency domain. In particular, a close approximation is developed that has comparable computation time to conventional SIFT but with improved matching performance. Results are presented for monocular wide-angle outdoor image sequences obtained using fisheye and equiangular catadioptric cameras. We evaluate the overall matching performance (recall versus 1-precision) of these methods compared to conventional SIFT. We also demonstrate the use of the technique for variable frame-rate visual odometry and its application to place recognition.
Agrawal, M. and Konolige, K. ( 2006). Real-time localization in outdoor environments using stereo vision and inexpensive GPS. International Conference on Pattern Recognition (ICPR), pp. 1063-1068.
2.
Bakstein, H. and Pajdla, T. ( 2002). Panoramic mosaicing with 180 degree field of view lens . Proceedings of the third Workshop on Omnidirectional Vision.
3.
Barreto, J. and Araujo, H. ( 2005). Geometric properties of central catadioptric line images and their application in calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8): 1327-1333.
4.
Basu, A. and Licardie, S. ( 1995). Alternate models for fish-eye lenses. Pattern Recognition Letters, 10: 433-441.
5.
Baumberg, A. ( 2000). Reliable feature matching across widely separated views . Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 774-781.
6.
Bay, H., Ess, A., Tuytelaars, T. and Van Gool, L. ( 2008). Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3): 346-359.
7.
Brown, M. and Lowe, D. ( 2002). Invariant features from interest point groups. Proceedings British Machine Vision Conference, Cardiff, Wales, pp. 656-665.
8.
Bülow, T. ( 2002). Multiscale image processing on the sphere. Proceedings of the 24th Symposium Pattern Recognition of the DAGM , van Gool, L. (ed.), pp. 609-617.
9.
Bülow, T. ( 2004). Spherical diffusion for 3D surface smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(12): 1650-1654.
10.
Bunschoten, R. and Kröse, B. ( 2003). Visual odometry from an omnidirectional vision system . Proceedings of the 2003 IEEE International Conference on Robotics and Automation, pp. 577-583.
11.
Chahl, S.J. and Srinivassan, V.M. ( 1997). Reflective surfaces for panoramic imaging. Applied Optics, 36(31): 8275- 8285.
12.
Civera, J., Davison, A. and Montiel, J. ( 2008). Interacting multiple model monocular SLAM. IEEE International Conference on Robotics and Automation (ICRA), pp. 3704- 3709.
13.
Claus, D. and Fitzgibbon, A.W. ( 2005). A rational function lens distortion model for general cameras. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
14.
Corke, P., Strelow, D. and Singh, S. ( 2004). Omnidirectional visual odometry for a planetary rover . Proceedings International Conference on Intelligent Robots and Systems, pp. 4007-4012.
15.
Cummins, M. and Newman, P. ( 2007). Probabilistic appearance based navigation and loop closing . IEEE International Conference on Robotics and Automation (ICRA), pp. 2042- 2048.
16.
Cummins, M. and Newman, P. ( 2008). FAB-MAP: Probabalistic localization and mapping in the space of appearance. The International Journal of Robotics Research, 27: 647- 665.
17.
Daniilidis, K., Makadia, A. and Bülow, T. ( 2002). Image processing in catadioptric planes: spaciotemporal derivatives and optical flow computation. Proceedings of the Third Workshop on Omnidirectional Vision.
18.
Daniilidis, K. and Nagel, H.-H. ( 1993). The coupling of rotation and translation in motion estimation of planar surfaces. Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pp. 188-193.
19.
Davison, A., Cid, Y.G. and Kita, N. ( 2004). Real-time 3D SLAM with wide-angle vision. IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Lisboa, Portugal.
20.
Davison, A., Reid, I., Molton, N. and Stasse, O. ( 2007). MonoSLAM: real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6): 1-16.
21.
Driscoll, J.R. and Healy, D.M. ( 1994). Computing fourier transforms and convolutions on the 2-sphere. Advances in Applied Mathematics, 15: 202-250.
22.
Fermüller, C. and Aloimonos, Y. ( 1998). Ambiguity in structure from motion: sphere versus plane . International Journal of Computer Vision, 28(2): 137-154.
23.
Fischler, M.A. and Bolles, R.C. ( 1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6): 381-395.
24.
Fleck, M.M. ( 1995). Perspective Projection: The Wrong Imaging Model. Technical Report Technical Report TR 95-01, Computer Science Department , University of Iowa.
25.
Geyer, C. and Daniilidis, K. ( 1999). Catadioptric camera calibration. Proceedings International Conference on Computer Vision, pp. 398-404.
26.
Geyer, C. and Daniilidis, K. ( 2000). A unifying theory for central panoramic systems and practical implications. Proceedings European Conference on Computer Vision.
27.
Geyer, C. and Daniilidis, K. ( 2001). Catadioptric projective geometry. International Journal of Computer Vision, 45(3): 223-243.
28.
Gluckman, J. and Nayar, S. ( 1998). Ego-motion and omnidirectional cameras. Proceedings International Conference on Computer Vision, pp. 999-1005.
29.
Groemer, H. ( 1996). Geometric Applications of Fourier Series and Spherical Harmonics. Cambridge, Cambridge University Press.
30.
Hansen, P., Corke, P., Boles, W. and Daniilidis, K. ( 2007a). Scale invariant feature matching with wide angle images . Proceedings of the International Conference on Intelligent Robots and Systems, pp. 1689-1694.
31.
Hansen, P., Corke, P., Boles, W. and Daniilidis, K. ( 2007b). Scale-invariant features on the sphere. Proceedings of the International Conference on Computer Vision , pp. 1-8.
32.
Hartley, R. and Zisserman, A. ( 2003). Multiple View Geometry in Computer Vision (2nd edn). Cambridge, Cambridge University Press.
33.
Ho, K.L. and Newman, P. ( 2007). Detecting loop closure with scene sequences. Interantional Journal of Computer Vision, 74(3): 261-286.
34.
Ho, T.-H., Davis, C.C. and Millner, S.D. ( 2005). Using geometric constraints for fisheye camera calibration . Proceedings of OMNIVIS 2005.
35.
Howard, A. ( 2008). Real-time stereo visual odometry for autonomous ground vehicles. Proceedings of the International Conference on Intelligent Robots and Systems.
36.
Hrabar, S. and Sukhatme, G. ( 2004). A comparison of two camera configurations for optic flow based navigation of a UAV through urban canyons. Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2673-2680.
37.
Hutchinson, S., Hager, G.D. and Corke, P.I. ( 1996). A tutorial on visual servo control. IEEE Transactions on Robotics and Autonomation12(5): 651-670.
38.
Jackson, J.D. ( 1975). Classical Electrodynamics, 2nd edition. New York, John Wiley & Sons.
39.
Johnson, A., Goldberg, S., Cheng, Y. and Matthies, L. ( 2008). Robust and efficient stereo feature tracking for visual odometry. Proceedings IEEE International Conference on Robotics and Automation (ICRA), pp. 39-46.
40.
Kadir, T. and Brady, M. ( 2001). Scale, saliency and image description. International Journal of Computer Vision, 45(2): 83-105.
41.
Kannala, J. and Brandt, S. ( 2004). A generic camera calibration method for fish-eye lenses . Proceedings International Conference on Pattern Recognition, pp. 10-13.
42.
Kannala, J. and Brandt, S. ( 2006). A generic camera model and calibration method for conventional, wide-angle and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8): 1335-1340.
43.
Ke, Y. and Sukthankar, R. ( 2004). PCA-SIFT: a more distinctive representation for local image descriptors. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 506-513.
44.
Kim, J. and Kweon, I.-S. ( 2007). Robust feature matching for loop closing and localization . IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3905-3910.
45.
Koenderink, J.J. ( 1984). The structure of images. Biological Cybernetics , 50: 363-370.
46.
Konolige, K., Agrawal, M. and Solà, J. ( 2007). Large-scale visual odometry for rough terrain. Proceedings of the International Symposium of Robotics Research .
47.
Lowe, D. ( 2004). Distinctive image features from scale-invariant keypoints . International Journal of Computer Vision, 60(2): 91-110.
48.
Lucas, B.D. and Kanade, T. ( 1981). An iterative image registration technique with an application to stereo vision. Proceedings of 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674-679.
49.
Maimone, M., Cheng, Y. and Matthies, L. ( 2007). Two years of visual odometry on the mars exploration rovers. Journal of Field Robotics, 24(3): 169-186.
50.
Marks, T., Howard, A., Bajracharya, M., Cottrell, G. and Matthies, L. ( 2008). Gamma-SLAM: using stereo vision and variance grid maps for SLAM in unstructured environments. IEEE International Conference on Robotics and Automation, pp. 3717-3724.
51.
Matas, J., Chum, O., Urban, M. and Pajdla, T. ( 2004). Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 22(10): 761-767.
52.
Mei, C. and Rives, P. ( 2007). Single view point omnidirectional camera calibration from planar grids. IEEE International Conference on Robotics and Automation, Roma, pp. 3945-3950.
53.
Mikolajczyk, K. and Schmid, C. ( 2004). Scale and affine invariant interest point detectors. International Journal of Computer Vision, 60(1): 63-86.
54.
Mikolajczyk, K. and Schmid, C. ( 2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence27(10): 1615-1630.
55.
Milford, M. and Wyeth, G. ( 2008). Mapping a suburb with a single camera using a biologically inspired SLAM system. IEEE Transactions on Robotics, 24(5): 1038-1053.
56.
Mičušík, B. (2004). Two-view Geometry of Omnidirectional Cameras. PhD Thesis, Czech Technical University, Centre for Machine Perception, Prague, Czech Republic.
57.
Mičušík, B. and Pajdla, T. (2003). Estimation of omnidirectional camera model from epipolar geometry. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition , pp. 485-490.
58.
Montemerlo, M., Thrun, S., Koller, D. and Wegbreit, B. ( 2002). FastSLAM: A factored solution to the simultaneous localization and mapping problem. Proceedings of the AAAI National Conference on Artificial Intelligence.
59.
Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F. and Sayd, P. ( 2006). Real time localization and 3D reconstruction. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 363-370.
60.
Nayar, S. and Baker, S. ( 1997). Catadioptric image formation. Proceedings DARPA Image Understanding Workshop.
61.
Nelson, C.R. and Aloimonos, J. ( 1988). Finding motion parameters from spherical motion fields (or the advantages of having eyes in the back of your head). Biological Cybernetics, 58: 261-273.
62.
Neumann, J., Fermüller, C. and Aloimonos, Y. ( 2002). Eyes form eyes: new cameras for structure from motion . Proceedings Workshop on Omnidirectional Vision (OMNIVIS).
63.
Nistér, D. ( 2000). Reconstruction from uncalibrated sequences with a hierarchy of trifocal tensors. Proceedings European Conference on Computer Vision, pp. 649-663.
64.
Nistér, D., Naroditsky, O. and Bergen, J. ( 2004). Visual odometry. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 652-659.
65.
Nistér, D., Naroditsky, O. and Bergen, J. ( 2006). Visual odometry for ground vehicle applications. Journal of Field Robotics, 23(1): 3-20.
66.
Ollis, M., Herman, H. and Singh, S. ( 1999). Analysis and Design of Panoramic Stereo Vision using Equi-angular Pixel Cameras. Technical Report CMU-RI-TR-99-04, Robotics Institute, Pittsburgh, PA.
67.
Piniés, P. and Tardós, J. ( 2007). Scalable SLAM building conditionally independent local maps. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3466-3471.
68.
Scaramuzza, D. and Siegwart, R. ( 2008). Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles. IEEE Transactions on Robotics , 24(5): 1015-1026.
69.
Schmid, C., Mohr, R. and Bauckage, C. ( 2000). Evaluation of interest point detectors. International Journal of Computer Vision, 37(2): 151-172.
70.
Sivic, J. and Zisserman, A. ( 2003). Video Google: a text retrieval approach to object matching in videos. Proceedings of the International Conference on Computer Vision, pp. 1470-1477.
71.
Stewénius, H., Engels, C. and Nistér, D. ( 2006). Recent developments on direct relative orientation. ISPRS Journal of Photogrammetry and Remote Sensing, 60(4): 284- 294.
72.
Streckel, B. and Koch, R. ( 2005). Lens Model Selection for Visual Tracking (Lecture Notes in Computer Science, Vol. 3663/2005). Berlin, Springer, pp. 41-48.
73.
Strelow, D. and Singh, S. ( 2004). Motion estimation from image and inertial measurements . The International Journal of Robotics Research, 23(12): 1157-1195.
74.
Swaminathan, R., Grossberg, M. and Nayar, S. ( 2003). A perspective on distortions. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 594-601.
75.
Tissainayagam, P. and Suter, D. ( 2004). Assessing the performance of corner detectors for point feature tracking applications. Image and Vision Computing , 22: 663-679.
76.
Tomasi, C. and Kanade, T. ( 1991). Detection and Tracking of Point Features. Technical Report CMU-CS-91-132, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.
77.
Tomono, M. ( 2007). Monocular SLAM using a raoblackwellised particle filter with exhaustive pose space search. IEEE International Conference on Robotics and Automation, pp. 2421-2426.
78.
Tuytelaars, T. and Van Gool, L. ( 2004). Matching widely separated views based on affine invariant regions. International Journal of Computer Vision, 59(1): 61-85.
79.
Vassallo, R.F., Santos-Victor, J. and Schneebeli, H.J. ( 2002). A genral approach for egomotion estiamtion with omnidirectional images. Proceedings of the Workshop on Omnidirectional Vision (OMNIVIS).
80.
Xiong, Y. and Turkowski, K. ( 1997). Creating image-based VR using a self-calibrating fisheye lens. Proceedings International Conference on Computer Vision and Pattern Recognition, pp. 237-243.
81.
Ying, X. and Hu, Z. ( 2004a). Can we consider central catiodioptric cameras and fisheye cameras within a unified imaging model. Proceedings of the 8th European Conference on Computer Vision, pp. 442-455.
82.
Ying, X. and Hu, Z. ( 2004b). Catadioptic camera calibration using geometric invariants . IEEE Transactions on Patteren Analysis and Machine Intelligence , 26(10): 1260- 1271.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
