We address the problem of vision-based navigation in busy inner-city locations, using a stereo rig mounted on a mobile platform. In this scenario semantic information becomes important: rather than modeling moving objects as arbitrary obstacles, they should be categorized and tracked in order to predict their future behavior. To this end, we combine classical geometric world mapping with object category detection and tracking. Object-category-specific detectors serve to find instances of the most important object classes (in our case pedestrians and cars). Based on these detections, multi-object tracking recovers the objects’ trajectories, thereby making it possible to predict their future locations, and to employ dynamic path planning. The approach is evaluated on challenging, realistic video sequences recorded at busy inner-city locations.
Ali, S. and Shah, M. ( 2008) Floor fields for tracking in high density crowd scenes . ECCV2008.
2.
Arras, K.O., Castellanos, J.A., Schilt, M. and Siegwart, R. ( 2003) Feature-based multi-hypothesis localization and tracking using geometric constraints. Robotics and Autonomous Systems , 44: 41-53.
3.
Arulampalam, M.S., Maskell, S., Gordon, N. and Clapp, T. ( 2002) A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing , 50: 174-188.
4.
Avidan, S. ( 2005) Ensemble tracking. CVPR2005.
5.
Badino, H., Franke, U. and Mester, R. ( 2007) Free space computation using stochastic occupancy grids and dynamic programming. ICCV Workshop on Dynamical Vision (WDV).
6.
Bajracharya, M., Moghaddam, B., Howard, A., Brennan, S. and Matthies, L.H. (2009) A fast stereo-based system for detecting and tracking pedestrians from a moving vehicle. The International Journal of Robotics Research, 28:1466-1485.
7.
Berclaz, J., Fleuret, F. and Fua, P. ( 2006) Robust people tracking with global trajectory optimization . CVPR2006.
8.
Betke, M., Haritaoglu, E. and Davis, L.S. ( 2000) Real-time multiple vehicle tracking from a moving vehicle . Machine Vision and Applications, 12(2): 69-83.
9.
Bibby, C. and Reid, I. ( 2008) Robust real-time visual tracking using pixel-wise posteriors . ECCV2008.
10.
Boros, E. and Hammer, P.L. ( 2002) Pseudo-Boolean optimization. Discrete Applied Mathematics, 123(1-3): 155-225.
11.
Breitenstein, M., Reichlin, F. and Van Gool, L. ( 2009) Robust tracking-by-detection using a detector confidence particle filter. ICCV2009.
12.
Cameron, S. and Proberdt, P. ( 1994) Advanced Guided Vehicles, Aspects of the Oxford AGV Project . Singapore, World Scientific.
13.
Comaniciu, D., Ramesh, V. and Meer, P. ( 2003) Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(5): 564-575.
14.
Cornelis, N. and Van Gool, L. ( 2005) Real-time connectivity constrained depth map computation using programmable graphics hardware. CVPR 2005, Vol. 1. Los Alamitos, CA, IEEE Computer Society Press, pp. 1099-1104.
15.
Cox, I.J. ( 1993) A review of statistical data association techniques for motion correspondence. The International Journal of Computer Vision, 10(1): 53-66.
16.
Dalal, N. and Triggs, B. ( 2005) Histograms of oriented gradients for human detection. CVPR2005.
Dollar, P., Wojek, C., Schiele, B. and Perona, P. ( 2009) Pedestrian detection: A benchmark. CVPR2009.
19.
Enzweiler, M. and Gavrila, D.M. ( 2009) Monocular pedestrian detection: Survey and experiments . IEEE Transactions on Pattern Analysis and Machine Intelligence .
20.
Ess, A., Leibe, B. and Van Gool, L. ( 2007) Depth and appearance for mobile scene analysis. ICCV 2007.
21.
Ess, A., Leibe, B., Schindler, K. and Van Gool, L. ( 2008) A mobile vision system for robust multi-person tracking . CVPR2008.
22.
Ess, A., Leibe, B., Schindler, K. and Van Gool, L. ( 2009a) Moving obstacle detection in highly dynamic scenes. ICRA2009.
23.
Ess, A., Leibe, B., Schindler, K. and Van Gool, L. ( 2009b) Robust multi-person tracking from a mobile platform. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(10): 1831-1846.
24.
Ess, A., Schindler, K., Leibe, B. and Van Gool, L. ( 2009c) Improved multi-person tracking with active occlusion handling. ICRA Workshop on People Detection and Tracking.
25.
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J. and Zisserman, A. ( 2008) The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results, http://www.pascal-network.org/challenges/VOC/voc2008/workshop/index.html .
26.
Felzenszwalb, P., McAllester, D. and Ramanan, D. ( 2008) A discriminatively trained, multiscale, deformable part model. CVPR2008.
27.
Felzenszwalb, P.F. and Huttenlocher, D.P. ( 2006) Efficient belief propagation for early vision. The International Journal of Computer Vision, 70: 41-54. Available at: http://people.cs.uchicago.edu/∼pff/bp/ .
28.
Fortmann, T.E., Bar Shalom, Y. and Scheffe, M. ( 1983) Sonar tracking of multiple targets using joint probabilistic data association. IEEE Journal of Oceanic Engineering, 8(3): 173-184.
29.
Gavrila, D.M. and Munder, S. ( 2007) Multi-cue pedestrian detection and tracking from a moving vehicle. The International Journal of Computer Vision, 73: 41-59.
30.
Gelb, A. ( 1996) Applied Optimal Estimation. Cambridge, MA, MIT Press.
31.
Grabner, H. and Bischof, H. ( 2006) On-line boosting and vision. CVPR2006.
32.
Isard, M. and Blake, A. ( 1998) CONDENSATION-conditional density propagation for visual tracking. The International Journal of Computer Vision, 29(1):5-28.
33.
Isard, M. and MacCormick, J. ( 2001) Bramble: A Bayesian multiple-blob tracker. ICCV 2001.
34.
Jiang, H., Fels, S. and Little, J.J. ( 2007) A linear programming approach for multiple object tracking . CVPR2007.
35.
Kaucic, R., Perera, A.G., Brooksby, G., Kaufhold, J. and Hoogs, A. ( 2005) A unified framework for tracking through occlusions and across sensor gaps. CVPR2005.
36.
Khan, Z., Balch, T. and Dellaert, F. ( 2005) MCMC-based particle filtering for tracking a variable number of interacting targets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11): 1805-1819.
37.
Lanz O. ( 2006) Approximate Bayesian multibody tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9): 1436-1449.
38.
Leibe, B., Cornelis, N., Cornelis, K. and Van Gool, L. ( 2007a) Dynamic 3D scene analysis from a moving vehicle. CVPR2007.
39.
Leibe, B., Leonardis, A. and Schiele, B. ( 2008a) Robust object detection with interleaved categorization and segmentation. International Journal of Computer Vision , 77(1-3): 259-289.
40.
Leibe, B., Schindler, K., Cornelis, N. and Van Gool, L. ( 2008b) Coupled detection and tracking from static cameras and moving vehicles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(10): 1683-1698.
41.
Leibe, B., Schindler, K. and Van Gool, L. ( 2007b) Coupled detection and trajectory estimation for multi-object tracking. ICCV 2007.
42.
Li, Y., Huang, C. and Nevatia, R. ( 2009) Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR2009.
43.
Morefield, C.L. ( 1977) Application of 0-1 integer programming to multitarget tracking problems. IEEE Transactions on Automatic Control .
44.
Munkres, J. ( 1957) Algorithms for the assignment and transportation problems . Journal of the Society of Industrial and Applied Mathematics , 5: 32-38.
45.
Murty, K.G. ( 1968) An algorithm for ranking all the assignments in order of increasing cost. Operations Research, 16: 682-687.
46.
Nillius, P., Sullivan, J. and Carlsson, S. ( 2006) Multi-target tracking-linking identities using Bayesian network inference. CVPR2006.
47.
Nistér, D., Naroditsky, O. and Bergen, J.R. ( 2004) Visual odometry. CVPR2004.
48.
Nummiaro, K., Koller-Meier, E. and Van Gool, L. ( 2003) An adaptive color-based particle filter. Image and Vision Computing, 21(1): 99-110.
49.
Okuma, K., Taleghani, A., de Freitas, N., Little, J. and Lowe, D. ( 2004) A boosted particle filter: multitarget detection and tracking . ECCV2004.
50.
Perera, A.G.A., Srinivas, C., Hoogs, A., Brooksby, G. and Hu, W. ( 2006) Multi-object tracking through simultaneous long occlusions and split-merge conditions. CVPR2006.
51.
Reid, D.B. ( 1979) An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24(6): 843-854.
52.
Rother, C., Kolmogorov, V., Lempitsky, V.S. and Szummer, M. ( 2007) Optimizing binary MRFs via extended roof duality. CVPR2007.
53.
Ryoo, M.S. and Aggarwal, J.K. ( 2008) Observe-and-explain: a new approach for multiple hypotheses tracking of humans and objects. CVPR2008.
54.
Schindler, K., James, U. and Wang, H. ( 2006) Perspective n-view multibody structure-and-motion through model selection. ECCV2006.
55.
Schrijver, A. ( 1998) Theory of Linear and Integer Programming. New York, John Wiley & Sons.
56.
Schulz, D., Burgard, W., Fox, D. and Cremers, A.B. ( 2001) Tracking multiple moving targets with a mobile robot using particle filters and statistical data association. ICRA2001.
57.
Song, X., Cui, J., Zha, H. and Zhao, H. ( 2008) Vision-based multiple interacting targets tracking via on-line supervised learning. ECCV2008.
58.
Stauffer, C. and Grimson, W.E.L. ( 1999) Adaptive background mixture models for real-time tracking . CVPR1999.
59.
Toyama, K., Krumm, J., Brumitt, B. and Meyers, B. ( 1999) Wall-flower: principles and practice of background maintenance . ICCV 1999.
60.
Viterbi A. ( 1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transactions on Information Theory, 13(2): 260-269.
61.
Wojek C., Schiele B. ( 2008) A dynamic CRF model for joint labeling of object and scene classes. ECCV2008.
62.
Wojek, C., Dorkó, G., Schulz, A. and Schiele, B. ( 2008) Sliding-windows for rapid object class localization: A parallel technique. DAGM2008.
63.
Wolf, J.K., Viterbi, A.M. and Dixson, G.S. ( 1989) Finding the best set of K paths through a trellis with application to multitarget tracking. IEEE Transactions on Aerospace and Electronic Systems, 25(29): 287-295.
64.
Wu, B. and Nevatia, R. ( 2007) Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet part detectors. The International Journal of Computer Vision, 75(2): 247-266.
65.
Yan, F., Kostin, A., Christmas, W.J. and Kittler, J. ( 2006) A novel data association algorithm for object tracking in clutter with application to tennis video analysis. CVPR2006.
66.
Yu, Q., Medioni, G. and Cohen, I. ( 2007) Multiple target tracking using spatio-temporal Markov chain Monte Carlo data association. CVPR2007 .
67.
Zach, C., Frahm, J.M. and Niethammer, M. ( 2009) Continuous maximal flows and Wulff shapes: Application to MRFs. CVPR2009.
68.
Zhang, L., Li, Y. and Nevatia, R. ( 2008) Global data association for multi-object tracking using network flows. CVPR2008.
69.
Zhao, T., Nevatia, R. and Wu, B. ( 2008) Segmentation and tracking of multiple humans in crowded environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(7): 1198-1211.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
