Sage Journals: Discover world-class research

Abstract

For mobile robotics, head gear in augmented reality (AR) applications or computer vision, it is essential to continuously estimate the egomotion and the structure of the environment. This paper presents the system developed in the SmartTracking project, which simultaneously integrates visual and inertial sensors in a combined estimation scheme. The sparse structure estimation is based on the detection of corner features in the environment. From a single known starting position, the system can move into an unknown environment. The vision and inertial data are fused, and the performance of both Unscented Kalman filter and Extended Kalman filter are compared for this task. The filters are designed to handle asynchronous input from visual and inertial sensors, which typically operate at different and possibly varying rates. Additionally, a bank of Extended Kalman filters, one per corner feature, is used to estimate the position and the quality of structure points and to include them into the structure estimation process. The system is demonstrated on a mobile robot executing known motions, such that the estimation of the egomotion in an unknown environment can be compared to ground truth.

Keywords

sensor fusion localization mapping

Get full access to this article

View all access options for this article.

References

Alves, J. , Lobo, J. , and Dias, J. (2003). Camera-inertial sensor modelling and alignment for visual navigation. IEEE International Conference on Advanced Robotics, Coimbra, Portugal, pp. 1693—1698.

Armesto, L. , Chroust, S. , Vincze, M. , and Tornero, J. (2004). Multi-rate fusion with vision and inertial sensors . IEEE International Conference on Robotics and Automation, pp. 193—199.

Armesto, L. , Tornero, J. , and Vincze, M. (2007). Fast egomotion estimation with multi-rate fusion of inertial and vision. International Journal of Robotics Research, 26(6): 577—589.

Chai, L. , Hoff, W. , and Vincent, T. (2002). 3-d motion and structure estimation using inertial sensors and computer vision for augmented reality. Presence, 11(5): 474—492.

Chandraker, M. , Stock, C. , and Pinz, A. (2003). Real-time camera pose in a room. 3rd International Conference on Computer Vision Systems, Graz, Austria, pp. 98—110.

Chen, J. and Pinz, A. (2004). Simultaneous tracking and modeling by fusion of inertial and vision sensors. Proceedings of the Workshop on Modelling and Motion Capture Techniques for Virtual Environments , pp. 26—31.

Chou, J. (1992). Quaternion kinematic and dynamic differential equations. IEEE Transactions on Robotics and Automation, 8(1): 53—64.

Chroust, S. and Vincze, M. (2004). Fusion of vision and inertia data for motion and structure estimation. Journal of Robotic Systems, 21(2): 73—83.

Corke, P. (2004). An inertial and visual sensing system for a small autonomous helicopter. Journal of Robotic Systems, 2(2): 43—51.

10.

Corke, P. , Lobo, J. , and Dias, J. (2007). An introduction to inertial and visual sensing . International Journal of Robotics Research, 26(6): 519—535.

11.

Davison, A. (2003). Real-time simultaneous localisation and mapping with a single camera. IEEE International Conference on Computer Vision, Nice, France, pp. 1403—1410.

12.

Davison, A. , Cid, Y. , and Kita, N. (2004). Real-time 3D SLAM with wide-angle vision. 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles, Lisboa, Portugal.

13.

Davison, A. , Reid, I. , Molton, N. , and Stasse, O. (2007) Monoslam: Real-time single camera SLAM. IEEE Transactions on Pattern Analysis and Machine Intelligence, to appear, 2007.

14.

Doucet, A. , de Freitas, J. , and Gordon, N. (2001). Sequential Monte Carlo Methods In Practice. Springer.

15.

Foxlin, E. and Naimark, L. (2003). Vis-tracker: A wearable vision-inertial selftracker . IEEE Virtual Reality, pp. 199— 206.

16.

Gemeiner, P. and Vincze, M. (2005). Motion and structure estimation from vision and inertial sensor data with high speed CMOS camera. IEEE International Conference on Robotics and Automation, Barcelona, Spain, pp. 1853—1858.

17.

Gemeiner, P. , Armesto, L. , Montés, N. , Tornero, J. , Vincze, M. , and Pinz, A. (2006). Visual tracking can recapture after fast motions with the use of inertial sensor. 30th Workshop of the Austrian Association for Pattern Recognition, Austria, pp. 141—150.

18.

Hartley, R. and Zisserman, A. (2003). Multiple View Geometry. Cambridge University Press.

19.

Huster, A. and Rock, S. (2001). Relative position estimation for interventioncapable AUVS by fusing vision and inertial measurements. International Symposium on Unmanned Unthethered Submersible Technology, Durban.

20.

Julier, S. , Uhlmann, J. , and Durrant-White, H. (2000). A new method for the nonlinear transformations of means and co-variances in filters and estimators. IEEE Transactions on Automation Control, 45: 477—482.

21.

Lobo, J. and Dias, J. (2003). Vision and inertial sensor cooperation using gravity as a vertical reference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12): 1597—1608.

22.

Lowe, D. , (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), pp. 91—110.

23.

Lu, C.-P. , Hager, G. , and Mjolsness, E. (2000). Fast and globally convergent pose estimation from video images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(6): 610—622.

24.

Mayer, P. , Edelmayer, G. , Gelderblom, G. , Vincze, M. , Einramhof, P. , Nuttin, M. , Fuxreiter, T. , and Kronreif, G. (2007). MOVEMENT — modular versatile mobility enhancement system. IEEE International Conference on Robotics and Automation, Rome, Italy, pp. 2892—2897.

25.

Mühlmann, U. , Ribo, M. , Lang, P. , and Pinz, A. (2004). A new high speed CMOS camera for real-time tracking applications. IEEE International Conference on Robotics and Automation, New Orleans, Vol. 5, pp. 5195—5200.

26.

Nistér, D. (2003). Preemptive RANSAC for live structure and motion estimation. IEEE International Conference on Computer Vision, Vol. 1, pp. 199—206.

27.

Nistér, D. , Naroditsky, O. , and Bergen, J. (2004). Visual odometry. IEEE International Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 652—659.

28.

Pollefeys, M. , Koch, R. , and Van Gool, L. (1999). Self-calibration and metric reconstruction inspite of varying and unknown intrinsic camera parameters. International Journal of Computer Vision, 32(1): 7—25.

29.

Rehbinder, H. and Ghosh, B. (2001). Multi-rate fusion of visual and inertial data . International Conference on Multi-Sensor Fusion and Integration for Intelligent Systems, pp. 97—102.

30.

Ribo, M. , Ganster, H. , Brandner, M. , Lang, P. , Stock, C. , and Pinz, A. (2002). Hybrid tracking for outdoor applications. IEEE Computer Graphics and Applications Magazine, 22(6): 54—63.

31.

Schweighofer, G. and Pinz, A. (2006). Robust pose estimation from a planar target. IEEE Transactions on Pattern Analysis and Maching Intelligence, 28(12): 2024—2030.

32.

St-Pierre, M. and Gingras, D. (2004). Comparison between the unscented kalman filter and the extended kalman filter for the position estimation module of an integrated navigation information system. IEEE Intelligent Vehicles Symposium, Parma, Italy, pp. 831—835.

33.

Thrun, S. , Burgard, W. , and Fox, D. (2005). Probabilistic Robotics. MIT Press, Cambridge, MA.

34.

Ude, A. ( 1999). Filtering in a unit quaternion space for model-based object tracking. Robotics and Autonomous Systems, 28(2-3): 163—172.

35.

XSens (2007). www.xsens.com .

36.

Zinsser, T. , Grässl, C. , and Niemann, H. (2005). High-speed feature point tracking. Workshop on Vision Modeling and Visualization, pp. 49—56.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

Simultaneous Motion and Structure Estimation by Fusion of Inertial and Vision Data

Abstract

Keywords

Get full access to this article

References

Supplementary Material