Towards Safer Roads by Integration of Road Scene Monitoring and Vehicle Control

Abstract

In this paper we introduce the Smart Cars Project at the Australian National University/National ICT Australia, together with a discussion and an example of a driver assistance system. We present a framework for interactive driver assistance systems that includes techniques for fast-speed sign detection and classification, obstacle detection and tracking applied to pedestrian detection, and lane departure warning. In addition, the driver’s actions are monitored. The integrated system uses information extracted from the road scene (speed signs, position within the lane, relative position to other cars, etc.) together with information about the driver’s state, such as eye gaze and head pose, to issue adequate warnings. A touch screen monitor is used to present relevant information and allow the driver to interact with the system. The research is focused around robust algorithms that are able to run on-line. Results of on-line speed sign detection and pedestrian detection are presented in the context of a driver assistance system.

Keywords

driver assistance systems sign detection and classification obstacle detection and tracking driver monitoring lane departure

Get full access to this article

View all access options for this article.

References

Apostoloff, N. and Zelinsky, A. 2002. Vision in and out of vehicles: integrated driver and road scene monitoring. Experimental Robotics VIII, Advanced Robotics Series, B. Siciliano and P. Dario , editors, Springer-Verlag, Berlin .

Arnell, F. and Petersson, L. 2005a. Fast object segmentation from a moving camera . Proceedings of the IEEE Intelligent Vehicle Symposium, Las Vegas, NV, June 6-8.

Arnell, F. and Petersson, L. 2005b. Generic fusion of visual cues applied to real-world object segmentation . Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Edmonton, Alberta, Canada, August 7-11.

Aufrere, R. , Gowdy, J. , Mertz, C. , Thorpe, C. , Wang, C-C. , and Yata, T. 2003. Perception for collision avoidance and autonomous driving . Mechatronics 13(10): 1149-1161 .

Australian Transport Safety Bureau (ATSB). 2004. Mortality and morbidity in Australia due to transport accidents. Available at http://www.atsb.gov.au/road/statistics/mortality_and_morbidity.aspx.

Bertozzi, M. , Broggi, A. , Carletti, M. , Fascioli, A. , Graf, T. , Grisleri, P. , and Meinecke, M. 2003. IR pedestrian detection for advanced driver assistance systems . Proceedings of the Pattern Recognition Symposium, Magdeburg, Germany, September.

Betke, M. and Makris, N. 1994. Fast object recognition in noisy images using simulated annealing. Technical Report, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, AIM-1510 . Available online at http://citeseer.ist.psu.edu/betke94fast.html.

Broggi, A. , Bertozzi, M. , and Fascioli, A. 2001a. Self-calibration of a stereo vision system for automotive applications . Proceedings of the IEEE International Conference on Robotics andAutomation, Seoul, Korea, May, pp. 3698-3702 .

Broggi, A. , Bertozzi, M. , Conte, G. , and Fascioli, A. 2001b. ARGO Prototype Vehicle. Intelligent Vehicle Technologies, L. Vlacic , F. Harashima , and M. Parent , editors, Butterworth-Heinemann, London , Chapter 14, pp. 445-493.

10.

Brooker, G. and Durrant-White, H. 2001. Millimetre wave radar . Proceedings of the Australian Conference on Robotics and Automation (ACRA), Melbourne, Australia.

11.

Cellario, M. 2001. Human-centered intelligent vehicles: Toward multimodal interface integration . Intelligent Transportation Systems 16(4): 78-81 .

12.

Coué, C. , Pradalier, C. , and Laugier, C. 2003. Bayesian programming formulti-target tracking: anautomotive application . Proceedings of the International Conference on Field and Service Robotics, Lake Yamanaka, Japan, July. Available on-line at http://emotion.inrialpes.fr/bibemotion/2003/CPL03.

13.

Dellaert, F. , Pomerleau, D. , and Thorpe, C. 1998. Model-based car tracking integrated with a road-follower . Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, May.

14.

Dickmanns, E. D. 1999. An expectation-based, multi-focal, saccadic (ems) vision system for vehicle guidance . Proceedings of the International Symposium on Robotics and Research, Salt Lake City, UT, October.

15.

Dickmanns, E. D. 2000. Vertebrate-type vision for autonomous vehicles . Proceedings of the Symposium on Biologically Inspired Systems, Wollongong, Australia, December.

16.

Dickmanns, E. D. and Zapp, A. 1987. Automonous high speed road vehicle guidance by computer vision . Proceedings of the International Federation of Automatic Control (IFAC) World Congress, Munich, Germany, July, pp. 221-226 .

17.

Fang, C. Y. , Fuh, C. S. , Chen, S. W. , and Yen, P. S. 2003. A road sign recognition system based on dynamic visual model . Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, June, Vol. 1, pp. 750-755 .

18.

Faugeras, O. , Hotz, B. , Mathieu, H. , Viéville, T. , Zhang, Z. , Fua, P. , Théron, E. , Moll, L. , Berry, G. , Vuillemin, J. , Bertin, P. , and Proy, C. 1993. Real-time correlation-based stereo: Algorithm, implementations and applications. Technical Report, French National Institute for Research in Computer Science and Control (INRIA), France .

19.

Fletcher, L. , Loy, G. , Barnes, N. , and Zelinsky, A. 2005. Correlating driver gaze with the road scene for driver assistance systems . Robotics and Autonomous Systems 52(1): 71-84 .

20.

Franke, U. and Heinrich, S. 2002. Fast obstacle detection for urban traffic situations . IEEE Transactions on Intelligent Transport Systems 3(3): 173-181 .

21.

Gavrila, D. M. 1998. A road sign recognition system based on dynamic visual model . Proceedings of the 14th International Conference on Pattern Recognition, Brisbane, Australia, August, Vol. 1, pp. 16-20 .

22.

Haworth, N. L. , Triggs, T. , and Grey, E. M. 1988. Driver fatigue: Concepts, measurement and crash countermeasures. Technical Report, Federal Office of Road Safety Contract Report 72 by Human Factors Group, Department of Psychology, Monash University .

23.

Hsu, S-H. and Huang, C-L. 2001. Road sign detection and recognition using matching pursuit method . Image and Vision Computing 19: 119-129 .

24.

Johansson, B. 2002. Road sign recognition from a moving vehicle. Master’s Thesis, Centre for Image Analysis, Swedish University of Agricultural Sciences.

25.

Kagami, S. , Okada, K. , Inaba, M. , and Inoue, H. 2000. Design and implementation of on body real-time depth map generation system . Proceedings of the IEEE International Conference on Robotics and Automation, San Francisco, CA, April 24-28.

26.

Kloeden, C. N. , McLean, A. J. , Moore, V. M. , and Ponte, G. 1997. Travelling speed and the risk of crash involvement. Technical Report, NHMRC Road Accident Research Unit, University of Adelaide . Available at http://casr.adelaide.edu.au/speed/.

27.

Labayrade, R. , Aubert, D. , and Tarel, J.-P. 2002. Real-time obstacle detection in stereovision on non-flat road geometry through v-disparity representation . Proceedings of the IEEE Intelligent Vehicle Symposium, Versailles, France, June.

28.

Land, M. and Lee, D. 1994. Where we look when we steer . Nature 369: 742-744 .

29.

Liu, Y. and Dingus, T. 1999. Development of human factors guidelines for advanced traveler information systems (ATIS) and commercial vehicle operations (CVO): Human factors evaluation of the effectiveness of multimodality dis-plays in advanced traveler information systems. Technical Report, Federal Highway Administration, Washington DC , FHWA-RD-96-150, July.

30.

Loy, G. and Zelinsky, A. 2003. Fast radial symmetry for detecting points of interest . IEEE Transactions on Pattern Analysis and Machine Intelligence 25(8): 959-973 .

31.

Loy, G. , Fletcher, L. , Apostoloff, N. , and Zelinsky, A. 2002. An adaptive fusion architecture for target tracking . Proceedings of the 5th International Conference on Automatic Face and Gesture Recognition, Washington, DC, May.

32.

Mertz, C. , McNeil, S. , and Thorpe, C. 2000. Side collision warning systems for transit buses . Proceedings of the IEEE Intelligent Vehicle Symposium (IV 2000), Dearborn, MI, October.

33.

Minor, L. G. and Sklansky, J. 1981. Detection and segmentation of blobs in infrared images . IEEE Transactions on Systems, Man, and Cybernetics 11(3): 194-201 .

34.

Miura, J. , Kanda, T. , and Shirai, Y. 2002. An active vision system for real-time traffic sign recognition . Proceedings of the IEEE Intelligent Vehicle Symposium, Dearborn, MI, October, pp. 52-57 .

35.

Paclik, P. , Novovicova, J. , Somol, P. , and Pudil, P. 2000. Road sign classification using the Laplace kernel classifier . Pattern Recognition Letters 21: 1165-1173 .

36.

Piccioli, G. , Micheli, E. D. , Parodi, P. , and Campani, M. 1996. Robust method for road sign detection and recognition . Image and Vision Computing 14(3): 209-223 .

37.

Priese, L. , Klieber, J. , Lakmann, R. , Rehrmann, V. , and Schian, R. 1994. New results on traffic sign recognition . Proceedings of the Intelligent Vehicle Symposium, Paris, France, August, pp. 249-254 . Available on-line at cite-seer.ist.psu.edu/priese94new.html.

38.

Roberts, J. and Corke, P. 2000. Obstacle detection for a mining vehicle using a 2D laser . Proceedings of the Australian Conference on Robotics and Automation (ACRA), Melbourne, Australia, August, pp. 185-190 .

39.

Shaposhnikov, D. G. , Podladchikova, L. N. , Golovan, A. V. , and Shevtsova, N. A. 2002. A road sign recognition system based on dynamic visual model . Proceedings of the 15th International Conference on Vision Interface, Calgary, Canada.

40.

Simoncelli, E. P. 1999. Bayesian Multiscale Differential Optical Flow, Academic, New York , Vol. 2, Chapter 14, pp. 297-422.

41.

Soto, A. and Khosla, P. 2001. Probabilistic adaptive agent based system for dynamic state estimation using multiple visual cues . Proceedings of the International Symposium on Robotics Research, Victoria, Australia, November.

42.

Strayer, D. L. and Johnson, W. A. 2001. Driven to distraction: Dual-task studies of simulated driving and conversing on a cellular telephone . Psychological Science 12(6): 462-466 .

43.

Sutherland, O. , Truong, H. , Rougeaux, S. , and Zelinsky, A. 2000. Advancing active vision systems by improved design and control . Proceedings of the International Symposium on Experimental Robotics (ISER2000), Waikiki, HI, December.

44.

Thomson, M. and Westland, S. 2001. Color-imager characterization by parametric fitting of sensor responses . Color Research and Application 26(6): 442-449 .

45.

Ulfvengren, P. 2003. Design of natural warning sounds in human-machine systems. Ph.D. Dissertation, Department of Industrial Economics and Management Series, Royal Institute of Technology, Stockholm, Sweden, December.

46.

Zhao, L. and Thorpe, C. 2000. Stereo and neural network-based pedestrian detection . IEEE Transactions on Intelligent Transportation Systems 1(3): 148-154 .