This paper is concerned with the problem of estimating the relative translation and orientation of an inertial measurement unit and a camera, which are rigidly connected. The key is to realize that this problem is in fact an instance of a standard problem within the area of system identification, referred to as a gray-box problem. We propose a new algorithm for estimating the relative translation and orientation, which does not require any additional hardware, except a piece of paper with a checkerboard pattern on it. The method is based on a physical model which can also be used in solving, for example, sensor fusion problems. The experimental results show that the method works well in practice, both for perspective and spherical cameras.
Bailey, T. and Durrant-Whyte, H. ( 2006). Simultaneous localization and mapping (SLAM): Part II . IEEE Robotics & Automation Magazine, 13(3): 108-117.
2.
Bay, H., Ess, A., Tuytelaars, T. and Van Gool, L. ( 2008). Speeded-up robust features (SURF). Journal of Computer Vision and Image Understanding, June, 346-359.
3.
Bleser, G. and Stricker, D. ( 2008). Advanced tracking through efficient image processing and visual-inertial sensor fusion. Proceedings of IEEE Virtual Reality Conference, Reno, NE, USA, March, pp. 137-144.
4.
Bouguet, J.-Y. ( 2003). Camera calibration toolbox for matlab, 2003. http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 2 April 2008.
5.
Chandaria, J., Thomas, G.A. and Stricker, D. ( 2007). The MA-TRIS project: real-time markerless camera tracking for augmented reality and broadcast applications. Journal of Real-Time Image Processing, 2(2): 69-79.
6.
Chatfield, A. ( 1997). Fundamentals of High Accuracy Inertial Navigation, vol. 174, 3rd edn. American Institute of Aeronautics and Astronautics, USA.
7.
Corke, P., Lobo, J. and Dias, J. ( 2007). An introduction to inertial and visual sensing. International Journal of Robotics Research, 26(6): 519-535.
8.
Daniilidis, K. ( 1999). Hand-eye calibration using dual quaternions. International Journal of Robotics Research, 18(3): 286-298.
9.
Durrant-Whyte, H. and Bailey, T. ( 2006). Simultaneous localization and mapping (SLAM): PartI. IEEE Robotics & Automation Magazine, 13(2): 99-110.
10.
Fischler, M.A. and Bolles, R.C. ( 1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6): 381-395.
11.
Foxlin, E. and Naimark, L. ( 2003). Miniaturization, calibration & accuracy evaluation of a hybrid self-tracker. Proceedings of 2nd International Symposium on Mixed and Augmented Reality, Tokyo, Japan, October, pp. 151-160.
12.
Graebe, S. ( 1990). Theory and implementation of gray box identification. Ph.D. Thesis, Royal Institute of Technology, Stockholm, Sweden.
13.
Harris, C. and Stephens, M. ( 1988). A combined corner and edge detector. Proceedings of the 4th Alvey Vision Conference, Manchester , UK, pp. 147-151.
14.
Hol, J.D. ( 2008). Pose estimation and calibration algorithms for vision and inertial sensors. Licentiate Thesis no 1379, Department of Electrical Engineering, Linköping University, Sweden.
15.
Hol, J.D., Schön, T.B. and Gustafsson, F. ( 2008a). A new algorithm for calibrating a combined camera and IMU sensor unit. Proceedings of 10th International Conference on Control, Automation, Robotics and Vision, Hanoi, Vietnam, December, pp. 1857-1862.
16.
Hol, J.D., Schön, T.B. and Gustafsson, F. ( 2008b). Relative pose calibration of a spherical camera and an IMU. Proceedings of 8th International Symposium on Mixed and Augmented Reality, Cambridge, UK, September, pp. 21-24.
17.
Horn, B.K.P. ( 1987). Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A, 4(4): 629-642.
18.
Kailath, T., Sayed, A.H. and Hassibi, B. ( 2000). Linear Estimation. Upper Saddle River, NJ, Prentice Hall.
19.
Kuipers, J.B. ( 1999). Quaternions and Rotation Sequences. Princeton, NJ, Princeton University Press.
20.
Ljung, L. ( 1999). System Identification: Theory for the User, 2nd edn. Upper Saddle River, NJ, Prentice Hall.
21.
Ljung, L. ( 2008). Perspectives on system identification. Proceedings of 17th International Federation of Automatic Control World Congress, Seoul, South Korea, July, pp. 7172-7184.
22.
Ljung, L. and Söderström, T. ( 1983). Theory and Practice of Recursive Identification. The MIT Press series in Signal Processing, Optimization, and Control . Cambridge, MA, MIT Press.
23.
Lobo, J. and Dias, J. ( 2007). Relative pose calibration between visual and inertial sensors. International Journal of Robotics Research, 26(6): 561-575.
24.
Lowe, D.G. ( 2004). Distinctive image features from scale-invariant keypoints . International Journal of Computer Vision, 60(2): 91-110.
25.
Magnus, J.R. and Neudecker, H. ( 1999). Matrix Differential Calculus with Applications in Statistics and Econometrics, 2nd edn. Chichester , UK, Wiley.
26.
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T. and van Gool, L. ( 2005). A comparison of affine region detectors. International Journal of Computer Vision, 65(1): 43-72.
27.
Mirzaei, F.M. and Roumeliotis, S.I. ( 2008). A Kalman filter-based algorithm for IMU-camera calibration: Observability analysis and performance evaluation. IEEE Transactions on Robotics, 24(5): 1143-1156.
28.
Nocedal, J. and Wright, S.J. ( 2006). Numerical Optimization. New York, Springer.
29.
Ozuysal, M., Fua, P. and Lepetit, V. ( 2007). Fast keypoint recognition in ten lines of code. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Minneapolis, MI, June, pp. 1-8.
30.
Scaramuzza, D., Martinelli, A. and Siegwart, R. ( 2006). A toolbox for easily calibrating omnidirectional cameras . Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, October, pp. 5695-5701.
31.
Shi, J. and Tomasi, C. ( 1994). Good features to track. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition , Seattle, WA, June, pp. 593- 600.
32.
Shuster, M.D. ( 1993). A survey of attitude representations. The Journal of the Astronautical Sciences, 41(4): 439-517.
33.
Titterton, D.H. and Weston, J.L. ( 1997). Strapdown inertial navigation technology. IEE Radar, Sonar, Navigation and Avionics Series. Stevenage, UK, Peter Peregrinus Ltd.
34.
Tsai, R.Y. and Lenz, R.K. ( 1989). A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Transactions on Robotics and Automation, 5(3): 345- 358.
35.
Woodman, O.J. ( 2007). An introduction to inertial navigation. Technical Report UCAM-CL-TR-696, University of Cambridge, Computer Laboratory.
36.
Zhang, Z. ( 2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11): 1330-1334.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
