This paper presents a novel approach to detect and track people and cars based on the combined information retrieved from a camera and a laser range scanner. Laser data points are classified by using boosted Conditional Random Fields, while the image based detector uses an extension of the Implicit Shape Model (ISM), which learns a codebook of local descriptors from a set of hand-labeled images and uses them to vote for centers of detected objects. Our extensions to ISM include the learning of object parts and template masks to obtain more distinctive votes for the particular object classes. The detections from both sensors are then fused and the objects are tracked using a Kalman Filter with multiple motion models. Experiments conducted in real-world urban scenarios demonstrate the effectiveness of our approach.
Ackermann, R. ( 1818). Improvements on axletrees applicable to four-wheeled carriages. Patent Number 4212.
2.
Arras, K.O., Grzonka, S., Luber, M. and Burgard, W. ( 2008). Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probabilities . IEEE International Conference on Robotics and Automation (ICRA) .
3.
Arras, K.O., Mozos, Ó.M. and Burgard, W. ( 2007). Using boosted features for the detection of people in 2D range data. Pasadena, USA IEEE International Conference on Robotics and Automation (ICRA). pp. 1710-1715
4.
Bar-Shalom, Y. and Li, X. ( 1995). Multitarget-Multisensor Tracking: Principles and Techniques . YBS Publishing.
5.
Belongie, S., Malik, J. and Puzicha, J. ( 2002). Shape matching and object recognition using shape contexts . IEEE Transactions on Pattern Analysis and Machine Intelligence , 24(4) 509-522.
6.
Borgefors, G. ( 1988). Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(6) 849-865.
7.
Bourgeois, F. and Lassalle, J.C. ( 1971). An extension of the Munkres algorithm for the assignment problem to rectangular matrices. Communications of the ACM , 14(12) 802-804.
8.
Comaniciu, D., Ramesh, V. and Meer, P. ( 2001). The variable bandwidth mean shift and data-driven scale selection. Vancouver, Canada In IEEE International Conference on Computer Vision (ICCV) pp. 438-445.
9.
Cox, I. and Hingorani, S.L. ( 2002). An efficient implementation of Reid’s multiple hypothesis tracking algorithm and its evaluation for the purpose of visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(2) 138-150.
10.
Cui, J., Zha, H., Zhao, H. and Shibasaki, R. ( 2005). Tracking multiple people using laser and vision . Edmonton, Canada IEEE International Conference on Intelligent Robotics and Systems (IROS) pp. 2116-2121.
11.
Dalal, N. and Triggs, B. ( 2005). Histograms of oriented gradients for human detection . San Diego, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 886-893.
12.
Douillard, B., Fox, D. and Ramos, F. ( 2008). Laser and vision based outdoor object mapping . Zurcih, Switzerland Robotics: Science and Systems (RSS).
13.
Enzweiler, M. and Gavrila, D. ( 2009). Monocular pedestrian detection: Survey and experiments . IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(12) 2179-2195.
14.
Felzenszwalb, P. and Huttenlocher, D. ( 2000). Efficient matching of pictorial structures. Hilton Head, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 66-73.
15.
Fod, A., Howard, A. and Matarić, M.J. ( 2002). A laser-based people tracker. Washington, DC, USA IEEE International Conference on Robotics and Automation (ICRA) pp. 3024-3029.
16.
Freund, Y. and Schapire, R.E. ( 1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1) 119-139.
17.
Gavrila, D. and Philomin, V. ( 1999). Real-time object detection for "smart" vehicles . Kerkrya, Greece IEEE International Conference on Computer Vision (ICCV) pp. 87-93.
18.
Hähnel, D., Triebel, R., Burgard, W. and Thrun, S. ( 2003). Map building with mobile robots in dynamic environments . Taipei, Taiwan IEEE International Conference on Robotics and Automation (ICRA) pp. 1557-1563.
19.
Ioffe, S. and Forsyth, D.A. ( 2001). Probabilistic methods for finding people. International Journal of Computer Vision, 43(1) 45-68.
20.
Li, K.-P. and Porter, J.P. ( 1988). Normalizations and selection of speech segments for speaker recognition scoring. New York, USA International Conference on Acoustics, Speech, and Signal Processing (ICASSP) pp. 595-598.
21.
Lafferty, J., McCallum, A. and Pereira, F. ( 2001). Conditional random fields: Probabilistic models for segmentation and labeling sequence data. International Conference on Machine Learning (ICML).
22.
Lau, B., Arras, K.O. and Burgard, W. ( 2009). Tracking groups of people with a multi-model hypothesis tracker. Kobe, Japan IEEE International Conference on Robotics and Automation (ICRA) pp. 3180-3185.
23.
Leibe, B., Cornelis, N., Cornelis, K. and Gool, L.V. ( 2007). Dynamic 3D scene analysis from a moving vehicle . Minneapolis, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1-8.
24.
Leibe, B., Mikolajczyk, K. and Schiele, B. ( 2006). Segmentation based multi-cue integration for object detection . Edinburgh, UK British Machine Vision Conference (BMVC).
25.
Leibe, B., Seemann, E. and Schiele, B. ( 2005). Pedestrian detection in crowded scenes. San Diego, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 878-885.
26.
Liu, D. and Nocedal, J. ( 1989). On the limited memory BFGS method for large scale optimization . Mathematical Programming, 45(3), (Series B) 503-528.
27.
Lloyd, S.P. ( 1982). Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2) 129-137.
28.
Luber, M., Arras, K.O., Plagemann, C. and Burgard, W. ( 2008). Classifying dynamic objects: An unsupervised learning approach. Zurich, Switzerland Robotics: Science and Systems (RSS).
29.
Mahalanobis, P. ( 1936). On the generalised distance in statistics. Proceedings of the National Institute of Science, India, volume 2.
30.
Mikolajczyk, K. and Schmid, C. ( 2005). A performance evaluation of local descriptors . IEEE Transactions on Pattern Analysis and Machine Intelligence , 27(10) 1615-1630.
31.
Navarro-Serment, L., Mertz, C. and Hebert, M. ( 2009). Pedestrian detection and tracking using three-dimensional ladar data. Boston, USA International Conference of Field and Service Robotics (FSR) pp. 102-112.
32.
Papageorgiou, C. and Poggio, T. ( 2000). A trainable system for object detection. International Journal of Computer Vision, 38(1) 15-33.
33.
Petrovskaya, A. and Thrun, S. ( 2008). Model based vehicle tracking for autonomous driving in urban environments. Zurich, Switzerland Robotics: Science and Systems (RSS).
34.
Pless, R. and Zhang, Q. ( 2004). Extrinsic calibration of a camera and laser range finder . Sendai, Japan IEEE International Conference on Intelligent Robotics and Systems (IROS) pp. 2301-2306.
Premebida, C., Ludwig, O. and Nunes, U. ( 2009). Lidar and vision-based pedestrian detection system . Journal of Field Robotics, 26(9) 696-711.
37.
Ramos, F., Fox, D., and Durrant-Whyte, H. (2007). CRFmatching: Conditional random fields for feature-based scan matching. Atlanta, USA Robotics: Science and Systems (RSS) 843-854.
38.
Reid, D. ( 1979). An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24(6) 843-854.
39.
Scheutz, M., Mcraven, J. and Cserey, G. ( 2004). Fast, reliable, adaptive, bimodal people tracking for indoor environments. Sendai, Japan IEEE International Conference on Intelligent Robotics and Systems (IROS) pp. 1347-1352.
40.
Schulz, D. ( 2006). A probabilistic exemplar approach to combine laser and vision for person tracking. Philadelphia, USA Robotics: Science and Systems (RSS).
41.
Schulz, D., Burgard, W., Fox, D. and Cremers, A. ( 2003). People tracking with mobile robots using sample-based joint probabilistic data ass. filters. International Journal of Robotics Research, 22(2) 99-116.
42.
Schwarz, G. ( 1978). Estimating the dimension of a model. Annals of Statistics, 6(2) 461-464.
43.
Sirovich, L. and Kirby, M. ( 1987). Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America . A, 4(3) 519-524.
44.
Spinello, L., Arras, K.O., Triebel, R. and Siegwart, R. ( 2010). A layered approach to people detection in 3D range data . Atlanta, USA Proceedings of the AAAI Conference on Artificial Intelligence.
45.
Spinello, L., Macho, A., Triebel, R. and Siegwart, R. ( 2009). Detecting pedestrians at very small scales. Pasadena, USA IEEE International Conference on Intelligent Robotics and Systems (IROS) pp. 3264-3269.
46.
Spinello, L. and Siegwart, R. ( 2008). Human detection using multimodal and multidimensional features. Nice, France IEEE International Conference on Robotics and Automation (ICRA) pp. 1823-1829.
47.
Spinello, L., Triebel, R. and Siegwart, R. ( 2008a). Multimodal detection and tracking of pedestrians in urban environments with explicit ground plane extraction. Chicago, USA IEEE International Conference on Intelligent Robotics and Systems (IROS).
48.
Spinello, L., Triebel, R. and Siegwart, R. ( 2008b). Multimodal people detection and tracking in crowded scenes. Proceedings of the AAAI Conference on Artificial Intelligence.
49.
Streller, D., Fuerstenberg, K.C. and Dietmaye, K. ( 2002). Vehicle and object models for robust tracking in traffic scenes using laser range images. Singapore, Singapore International Conference on International Transport Systems (ITSC) pp. 118-123.
50.
Topp, E.A. and Christensen, H.I. ( 2005). Tracking for following and passing persons. Edmonton, Canada IEEE International Conference on Intelligent Robotics and Systems (IROS) pp. 2321-2327.
51.
Viola, P., Jones, M.J. and Snow, D. ( 2003). Detecting pedestrians using patterns of motion and appearance . IEEE International Conference on Computer Vision (ICCV).
52.
Wender, S. and Dietmayer, K. ( 2008). 3D vehicle detection using a laser scanner and a video camera. Intelligent Transport Systems (IET)2 (2): 105-112.
53.
Xavier, J., Pacheco, M., Castro, D., Ruano, A. and Nunes, U. ( 2005). Fast line, arc/circle and leg detection from laser scan data in a player driver. Barcelona, Spain IEEE International Conference on Robotics and Automation (ICRA) pp. 3930-3935.
54.
Zhang, Z. ( 1999). Flexible camera calibration by viewing a plane from unknown orientations. Kerkyra, Greece IEEE International Conference on Computer Vision (ICCV) pp. 666-673.
55.
Zhao, L. and Thorpe, C. ( 1998). Qualitative and quantitative car tracking from a range image sequence. Santa Barbara, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 496-501.
56.
Zheng, W. and Liang, L. ( 2009). Fast car detection using image strip features . Miami, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2703-2710.
57.
Zhu, Q., Yeh, M.C., Cheng, K.T. and Avidan, S. ( 2006). Fast human detection using a cascade of histograms of oriented gradients. New York, USA IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1491-1498.
58.
Zivkovic, Z. and Kröse, B. ( 2007). Part based people detection using 2D range data and images . San Diego, USA. IROS . pp. 214-219.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
