Visual Map-Based Localization for Intelligent Vehicles Using Around View Monitoring in Underground Parking Lots

Abstract

Accurate and robust self-localization is a crucial task for intelligent vehicles. Because of limited access to GPS signals, localization in underground parking lots remains a problem. In this paper, fusion localization for intelligent vehicles using the widely available around view monitoring (AVM) is conducted by Kalman filter based on second-order Markov motion model (KF-MM2). The proposed method consists of two steps, one for visual map construction from AVM images and the other for map-based multi-scale localization. The proposed visual map consists of a series of nodes. Each node encodes both holistic and local visual features computed from AVM images, three-dimensional structure, and vehicle pose. In the localization step, the process of image-level localization is modeled as a Hidden Markov Model (HMM), in which the map nodes are hidden states. The result of image-level localization is calculated using forward algorithm by the given AVM image sequence. Then the metric localization is computed from local features matching. Finally, the metric localization is fused with the prediction by KF-MM2. The proposed method has been verified in two typical underground parking lots. Experimental results demonstrate that the proposed method can achieve an average error of 0.39 m in underground parking lots.

Keywords

intelligent vehicles visual map around view system Kalman filter Hidden Markov Model

Get full access to this article

View all access options for this article.

References

Kuutti

Fallah

Katsaros

Dianati

Mccullough

Mouzakitis

A Survey of the State-of-the-Art Localization Techniques and Their Potentials for Autonomous Vehicle Applications. IEEE Internet of Things Journal, Vol. 5, No. 2, 2018, pp. 829–846.

Zhong

Guo

Yang

On Real Time Performance Evaluation of the Inertial Sensors for INS/GPS Integrated Systems. IEEE Sensors Journal, Vol. 16, No. 17, 2016, pp. 6652–6661.

Nguyen

Nashashibi

Thanh-Huong

Castelli

Indoor Intelligent Vehicle Localization Using WiFi Received Signal Strength Indicator. Proc., 2017 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM), Nagoya, Japan, IEEE, New York, 2017, pp. 33–36.

Poulose

Han

D. S.

UWB Indoor Localization Using Deep Learning LSTM Networks. Applied Sciences-Basel, Vol. 10, No. 18, pp. 6290–6313.

Jian

Gang

Zheng

Shi

An INS/WiFi Indoor Localization System Based on the Weighted Least Squares. Sensors, Vol. 18, No. 5, 2018, art no. 1458.

Huang

Wang

Zhang

, Multi-View and Multi-Scale Localization for Intelligent Vehicles in Underground Parking Lots. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 791–800.

Tao

Huang

Cai

Wang

LiDAR-Only Vehicle Localization Based on Map Generation. Transportation Research Board, Washington, D.C., 2020, pp. 1–14.

Zhang

Sanjiv

LOAM: LiDAR Odometry and Mapping in Real-Time. Robotics: Science and Systems, Vol. 2, 2014, pp. 1–10.

Wan

Yang

Cai

Zhou

Wang

Song

Robust and Precise Vehicle Localization Based on Multi-Sensor Fusion in Diverse City Scenes. Proc., IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, IEEE, New York, 2018, pp. 4670–4677.

10.

Xiang

Liu

High-Performance Visual Odometry With Two-Stage Local Binocular BA and GPU. Proc., IEEE Intelligent Vehicles Symposium (IV), Gold Coast, QLD, Australia. IEEE, New York, 2013, pp. 1107–1112.

11.

Murartal

Tardos

J. D.

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras. IEEE Transactions on Robotics, Vol. 33, No. 5, 2017, pp. 1255–1262.

12.

Cai

Sotelo

M. A.

Visual Map-Based Localization for Intelligent Vehicles From Multi-View Site Matching. IEEE Transactions on Intelligent Transportation Systems, Vol. 22, No. 2, 2021, pp. 1068–1079.

13.

Zhang

Liu

Tian

A Coarse to Fine Indoor Visual Localization Method Using Environmental Semantic Information. IEEE Access, Vol. 7, 2019, pp. 21963–21970.

14.

Milford

M. J.

Wyeth

G. F.

SeqSLAM: Visual Route-Based Navigation for Sunny Summer Days and Stormy Winter Nights. Proc., IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN. IEEE, New York, 2012, pp. 1643–1649.

15.

Deng

Yang

Wang

Semantic Segmentation-Based Lane-Level Localization Using Around View Monitoring System. IEEE Sensors Journal, Vol. 19, No. 21, 2019, pp. 10077–10086.

16.

Huang

Zhao

Zhang

Feng

Vision-Based Semantic Mapping and Localization for Autonomous Indoor Parking. Proc., IEEE Intelligent Vehicles Symposium (IV), Changshu, China, IEEE, New York, 2018, pp. 636–641.

17.

Kim

B. J.

Chung

Lane-Level Localization Using an AVM Camera for an Automated Driving Vehicle in Urban Environments. IEEE-ASME Transactions on Mechatronics, Vol. 22, No. 1, 2017, pp. 280–290.

18.

Rublee

Rabaud

Konolige

Bradski

ORB: An Efficient Alternative to SIFT or SURF. Proc., International Conference on Computer Vision (ICCV), Barcelona, Spain, IEEE, New York, 2011, pp. 2564–2571.

19.

Calonder

Lepetit

Strecha

Fua

BRIEF: Binary Robust Independent Elementary Features. Proc., European Conference on Computer Vision (ECCV), Crete, Greece, 2010, pp. 778–792.

20.

Geiger

Lenz

Stiller

Urtasun

Vision Meets Robotics: The KITTI Dataset. International Journal of Robotics Research, Vol. 32, No. 11, 2013, pp. 1231–1237.

21.

Fischler

M. A.

Bolles

R. C.

Random Sample Consensus: A Paradigm for Model Fitting With Applications to Image Analysis and Automated Cartography. Communications of the ACM, Vol. 24, No. 6, 1981, pp. 381–395.

22.

Bang

Pyo

Jeong

AVM Auto-Calibration for Driving Vehicle Images. In Information Science and Applications (ICISA) ( Kim

Joukov

, eds.), Springer, Singapore, 2016, pp. 471–480.

23.

Cummins

Newman

FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance. International Journal of Robotics Research, Vol. 27, No. 6, 2008, pp. 647–665.