Multiple-Camera Tracking of Rigid Objects

Abstract

In this paper we describe a method for tracking rigid objects using one or several cameras. The tracking process consists of aligning a 3-D model representation of an object with image contours by measuring and minimizing the image error between predicted model points and image contours. The tracker behaves like a visual servo loop where the internal and external camera parameters are updated at each new image acquisition. We study in detail the Jacobian matrix associated with this minimization process in the presence of both point-to-point and point-to-contour matches. We establish the minimal number of matches that are needed as well as the singular configurations leading to a rank-deficient Jacobian matrix. We find a mathematical link between the point-to-point and point-to-contour cases. Based on this link we show that the latter has the same kind of singularities as the former. Moreover, we study multiple-camera configurations which optimize the robustness of the method in the presence of single-camera singularities, bad, noisy, or missing data. Extensive experiments done with a complex ship part and with up to three cameras validate the method. In particular we show that the tracker may well be used as a camera calibration procedure.

Keywords

object tracking visual servoing camera calibration

Get full access to this article

View all access options for this article.

References

Armstrong, M. , and Zisserman, A. 1995. Robust object tracking. In Proc. Asian Conference on Computer Vision, Vol. I, pp. 58-61.

Delamarre, Q. , and Faugeras, O. 1999. 3D articulated models and multi-view tracking with silhouettes . In Proc. 7th IEEE Int. Conf. on Computer Vision , Vol. 2, Corfu, Greece, 20- 25 September, pp. 716-721.

Drummond, T. , and Cipolla, R. 1999a. Real-time tracking of complex structures for visual servoing . In Proc. 7th IEEE Int. Conf. on Computer Vision , Corfu, Greece, September.

Drummond, T. , and Cipolla, R. 1999b. Real-time tracking of complex structures for visual servoing . In Vision Algorithms, Theory and Practice, pp. 69-83. Springer Verlag, LNCS.

Drummond, T. , and Cipolla, R. 1999c. Real-time tracking of complex structures with on-line camera calibration. In Proc. British Machine Vision Conf, Vol. 2, Nottingham, UK, September, pp. 574-583.

Drummond, T. , and Cipolla, R. 2000. Real-time tracking of multiple articulated structures in multiple views. In Proc. 6th European Conf. on Computer Vision, Vol. 2, Dublin, Ireland, June-July, pp. 20-36.

Espiau, B. 1993. Effect of camera calibration errors on visual servoing in robotics. In Proc. 3rd Int. Symp. on Experimental Robotics, Kyoto, Japan, October, pp. 187-193.

Faugeras, O.D. 1993. Three Dimensional Computer Vision: A Geometric Viewpoint . Boston: MIT Press.

Harris, C. , and Stennett, C. 1990. RAPID-A video rate object tracker. Proceedings of the First British Machine Vision Conference, Oxford, September, pp. 73-78.

10.

Hartley, R. , and Zisserman, A. 2000. Multiple View Geometry in Computer Vision. Cambridge, UK: Cambridge University Press.

11.

Horaud, R. , Csurka, G. , and Demirdjian, D. 2000. Stereo calibration from rigid motions. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12):1446-1452.

12.

Hutchinson, S. , Hager, G.D. , and Corke, P.I. 1996. A tutorial on visual servo control. IEEE Transactions on Robotics and Automation 12(5):651-670.

13.

Kinoshita, K. , and Deguchi, K. 1994. Simultaneous determination of camera pose and intrinsic parameters by visual servoing. Proc. of the 12th IAPR International Conference on Pattern Recognition 1:285-289.

14.

Lamiroy, B. , Drummond, T. , Horaud, R. , and Knudsen-Neckelman, O. 2000a. Visually guided robots for ship building. 1st International Conference on Computer Applications and Information Technology in the Maritime Industries (COMPIT'2000), Potsdam/Berlin, Germany, March 30-April 2.

15.

Lamiroy, B. , Espiau, B. , Andreff, N. , and Horaud, R. 2000b. Controlling robots with two cameras: how to do it properly . Proc. of IEEE International Conference on Robotics and Automation, San Francisco, CA, April, pp. 2100-2105.

16.

Lowe, D. 1991. Fitting parameterized three-dimensional models to images . IEEE Transactions on Pattern Analysis and Machine Intelligence 13(5):441-450.

17.

Lowe, D. 1992. Robust model-based motion tracking through the integration of search and estimation. International Journal of Computer Vision 8(2):113-122.

18.

Luong, Q.-T. , and Faugeras, O.D. 2001. The Geometry of Multiple Images. Boston : MIT Press.

19.

Marchand, E. , Bouthemy, P. , Chaumette, F. , and Moreau, V. 1999. Robust real-time visual tracking using a 2D-3D model-based approach. In Proc. 7th IEEE International Conference on Computer Vision, Vol. 1, Corfou, Greece, September, pp. 262-268.

20.

Phong, T.Q. , Horaud, R. , Yassine, A. , and Pham, D.T. 1995. Object pose from 2-D to 3-D point and line correspondences . International Journal of Computer Vision 15(3):225-243. Kluwer Academic Publishers.

21.

Rasmussen, C. , and Hager, G.D. 2001. Probabilistic data association methods for tracking complex visual objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(6):560-576.

22.

Thompson, R.L. , Reid, I.D. , Munoz, L.A. , and Murray, D.W. 2001. Providing synthetic views for teleoperation using visual pose tracking in multiple caeras. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 1(31):43-53.

23.

Tonko, M. , and Nagel, H.-H. 2000. Model-based stereo-tracking of non-polyhedral objects for automatic disassembly experiments. International Journal of Computer Vision 37(1):99-118.

24.

Vincze, M. , Ayromlou, M. , Ponweisser, W. , and Zillich, M. 2001. Edge-projected integration of image and model cues for robust model-based object tracking. International Journal of Robotics Research 20(7):533-552.