Location recognition of unmanned vehicles based on visual semantic information and geometric distribution

Abstract

Due to the influence of weather, light, season and angle changes, the appearance of objects changes visually, which makes it difficult for unmanned vehicles that rely on visual positioning to complete their positioning work. This paper proposes a coordinated positioning method that is composed of semantic information and geometric relationships distribution (GRD), which improves the robustness of unmanned vehicle location under the above conditions. First, we improved the FAST-SCNN semantic segmentation network and replaced its fully connected layer with the conv4-3 module to prevent the spatial information of the image from being lost in the fully connected layer. At the same time, the conv4-3 layer contains the richest semantic information, we use image semantic content to create a dense and prominent scene description. These prominent descriptions were learned from a large data set of perceptual changes. The method can accurately segment geometrically stable image regions. We combine the characteristics of these highlighted areas with the existing overall representation to produce a more robust scene descriptor. Second, a method is designed to integrate the matching of semantic labels and geometric distribution relations, which is a new closed loop location recognition label and landmark map. The geometric pair relationship between the ground marks is encoded as a continuous probability density function, the GRD function, which is expressed by a Laguerre polynomial and Fourier series basis expansion. This orthogonal basis representation allows for efficient computation of rotation and translation invariants, which are used to compare signatures and search for potential loop closure candidates. Finally, we evaluate our method with some of the most advanced algorithms, such as OpenSeqSLAM, AlexNet and VSO, to demonstrate its advantages. The experimental results for representative data sets show that the method based on Fast-SCNN is superior to other methods.

Keywords

Location recognition semantic information geometric relationships distribution FAST-SCNN

Get full access to this article

View all access options for this article.

References

Zhou

Lapedriza

Khosla

, et al. Places: a 10 million image database for scene recognition. IEEE Trans Pattern Anal Mach Intell 2018; 40(6): 1452–1464.

Zaffar

Ehsan

Milford

, et al. CoHOG: a light-weight, compute-efficient, and training-free visual place recognition technique for changing environments. IEEE Robot Autom Lett 2020; 5: 1835–1842.

Chancán

Hernandez-Nunez

Narendra

, et al. A hybrid compact neural architecture for visual place recognition. IEEE Robot Autom Lett 2020; 5: 993–1000.

Lowry

Sünderhauf

Newman

, et al. Visual place recognition: a survey. IEEE T Robot 2016; 32: 1–19.

Hochreiter

Schmidhuber

. Long short-term memory. Neural Comput 1997; 9: 1735–1780.

Banino

Barry

Uria

, et al. Vector-based navigation using grid-like representations in artificial agents. Nature 2018; 557: 429–433.

Cueva

Wei

. Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. arXiv:1803.07770. 2018.

Pei

Deng

Song

, et al. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 2019; 572: 106–111.

Yang

Wang

, et al. DashNet: a hybrid artificial and spiking neural network for high-speed object tracking. arXiv:1909.12942. 2019.

10.

McManus

Upcroft

Newmann

. Scene signatures: localised and point-less features for localisation. In: Fox

Kavraki

Kurniawati

. (eds) Robotics: science and systems. Cambridge, MA: MIT Press, 2014, pp.1–9.

11.

Chen

Obadiah

Jacobson

, et al. Convolutional neural network based place recognition. arXiv:1411.1509. 2014.

12.

Sünderhauf

Dayoub

Shirazi

, et al. On the performance of ConvNet features for place recognition. In: 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS), Hamburg, Germany, 28 September–2 October 2015, New York: IEEE.

13.

Sunderhauf

Shirazi

Jacobson

, et al. Place recognition with ConvNet landmarks: Viewpoint-robust, condition-robust, training-free. In: Hsu

. (ed.) Robotics: science and systems XII. Springer, 2015, pp.1–10.

14.

Gronat

Obozinski

Sivic

, et al. Learning and calibrating per-location classifiers for visual place recognition. In: Proceeding of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, 23–28 June 2013, pp.907–914. IEEE.

15.

Himstedt

Frost

Hellbach

, et al. Large-scale place recognition in 2-D LIDAR scans using geometrical landmark relations. In: 2014 IEEE/RSJ international conference on intelligent robots and systems, Chicago, IL, 14–18 September 2014, pp.5030–5035. New York: IEEE.

16.

Qin

Zhang

Liu

, et al. Appearance-invariant place recognition by adversarially learning disentangled representation. Robot Auton Syst 2020; 131: 103561.

17.

Lopez-Antequera

Gomez-Ojeda

Petkov

, et al. Appearance-invariant place recognition by discriminatively training a convolutional neural network. Pattern Recognit Lett 2017; 92: 89–95.

18.

Zhu

Tian

, et al. Visual place recognition in long-term and large-scale environment based on CNN feature. In: 2018 IEEE intelligent vehicles symposium (IV), Changshu, China, 26–30 June 2018. New York: IEEE.

19.

Sünderhauf

Dayoub

Shirazi

20.

Naseer

Oliveira

Brox

, et al. Semantics-aware visual localization under challenging perceptual conditions. In: 2017 IEEE international conference on robotics & automation, Singapore, 29 May–3 June 2017. New York: IEEE.

21.

Razavian

Azizpour

Sullivan

, et al. CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPR), Columbus, Ohio, 17–24 June 2014, pp.806–813. New York: IEEE.

22.

Hou

Zhang

Zhou

. BoCNF: efficient image matching with Bag of ConvNet features for scalable and robust visual place recognition. AutonRobots 2018; 42: 1169–1185.

23.

Arandjelovic

Gronat

Torii

, et al. NetVLAD: CNN architecture for weakly supervised place recognition. IEEE Trans Pattern Anal Mach Intell. Epub ahead of print 1 June 2017. DOI: 10.1109/TPAMI.2017.2711011.

24.

Bowman

Atanasov

Daniilidis

, et al. Probabilistic data association for semantic SLAM. In: 2017 IEEE international conference on robotics and automation (ICRA), Singapore, 29 May–3 June 2017, New York: IEEE.

25.

Lianos

Schönberger

Pollefeys

, et al. VSO: visual semantic odometry. In: Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018, pp.246–263. Cham: Springer.

26.

Talbot

Garg

Milford

. OpenSeqSLAM2.0: an open source toolbox for visual place recognition under changing conditions. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, Spain, 1–5 October 2018, pp.7758–7765. New York: IEEE.

27.

Milford

Wyeth

. SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: 2012 IEEE international conference on robotics and automation, Saint Paul, MN, 14–18 May 2012, pp.1643–1649. New York: IEEE.

28.

Stenborg

Toft

Hammarstrand

Long-term visual localization using semantically segmented images. In: Proceedings of the 2018 IEEE international conference on robotics and automation (ICRA), Brisbane, QLD, 21–25 May 2018, pp. 6484–6490. New York: IEEE.

29.

Mousavian

Košecká

Lien

. Semantically guided location recognition for outdoors scenes. In: Proceedings of the IEEE international conference on robotics and automation (ICRA), Seattle, WA, 26–30 May 2015. New York: IEEE.

30.

Csurka

Dance

Fan

, et al. Visual categorization with bags of keypoints. In: ECCV Workshop statistical learning in computer vision, Prague, Czech, 11–14 May 2004, pp.1–22. Springer.

31.

Deray

Solà

Andrade-Cetto

. Word ordering and document adjacency for large loop closure detection in 2-D laser maps. IEEE Robot Autom Lett Jul 2017; 2: 1532–1539.

32.

Liu

Petillot

Lane

, et al. Global localization with object-level semantics and topology. In: 2019 international conference on robotics and automation (ICRA), Montreal, QC, 20–24 May 2019. New York: IEEE.

33.

Himstedt

Frost

Hellbach

34.

Himstedt

Maehle

Geometry matters: place recognition in 2-D range scans using geometrical surface relations. In: 2015 european conference on mobile robots (ECMR), Lincoln, 2–4 September 2015, pp.1–6. New York: IEEE.

35.

Bai

Wang

Zhang

, et al. CNN feature boosted SeqSLAM for real-time loop closure detection. arXiv:1704.05016. 2017.

36.

Liu

Rabinovich

Berg

. ParseNet: looking wider to see better. arXiv: 1506.04579. 2016.

37.

Long

Shelhamer

Darrell

. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision and pattern recognition (CVPR), Boston, 8–10 June 2015, pp.3431–3440. New York: IEEE.

38.

Girshick

Donahue

Darrell

, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE international conference on computer vision and pattern recognition (CVPR), Columbus, OH, USA, 23–28 June 2014, pp.580–587. IEEE.

39.

Kallasi

Rizzini

. Efficient loop closure based on FALKOLIDAR features for online robot localization and mapping. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), Daejeon, 9–14 October 2016, pp.1206–1213. New York: IEEE.