In visual-inertial odometry (VIO) research, combining point and line features can satisfy high requirements of integrity and restoration for scene reconstruction. In this paper, an integrating strategy of two types of line segments is proposed on the basis of utilizing point and line features for VIO frameworks. First, line segments are classified as short and long categories after extraction and tracking to enhance feature utilization. Then, short lines are spliced and long lines are merged on the basis of distance constraint and consistent direction. In the meanwhile, considering epipolar constraints during back-end processing, triangulation of the two line categories is optimized on the premise of correct matching between adjacent frames, so as to improve accuracy of pose estimation. Finally, point and line feature constraints are utilized simultaneously to reconstruct environments with back-end optimization. Evaluation on the EuRoC dataset shows that localization accuracy and robustness are improved with the proposed strategy.1
AkinlarCTopalC (2011) EDLines: A real-time line segment detector with a false detection control. Pattern Recognition Letters32(13): 1633−1642.
2.
BadalkhaniSHavangiRFarshadM (2021) An improved simultaneous localization and mapping for dynamic environments. International Journal of Robotics and Automation36(6): 374−382, 2021.
3.
BartoliASturmP. (2016) Structure-from-motion using lines: Representation, triangulation, and bundle adjustment. Computer Vision and Image Understanding100(3): 416-441.
4.
BurriMNikolicJSiegwartR (2016) The EuRoC micro aerial vehicle datasets. International Journal of Robotics Research35(10): 1157–1163.
5.
BazinJPollefeysM (2012) 3-line RANSAC for orthogonal vanishing point detection. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal, 07-12 October, pp. 4282−4287. New York: IEEE.
6.
CamposecoFPollefeysM (2015) Using vanishing points to improve visual-inertial odometry. In: 2015 IEEE International Conference on Robotics and Automation, Seattle, WA, USA, 26-30 May, pp. 5219−5225. New York: IEEE.
7.
FengLZhangXPengX, et al. (2024) Improved monocular visual-inertial odometry with point and line features using adaptive line feature extraction. Multimedia Tools and Applications83(1): 1481-1512.
8.
ForsterCCarloneLDellaertF, et al. (2016) On-manifold preintegration for real-time visual–inertial odometry. IEEE Transactions on Robotics33(1): 1–21.
9.
FuQWangJYuH, et al. (2020) PL-VINS: Real-time monocular visual-inertial SLAM with point and line features. arXiv preprint arXiv 07462.
10.
Gomez-OjedaRMorenoFZuñiga-NoëlDPL-SLAM: A stereo SLAM system through the combination of points and line segments. IEEE Transactions on Robotics35(3): 734-746.
11.
Galvez-LopezDTardosJ (2011) Real-time loop detection with bags of binary words. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25-30 September, pp. 51−58. New York: IEEE.
12.
HartleyRZissermanA (2003) Multiple view geometry in computer vision. Cambridge: Cambridge University Press.
13.
HeYZhaoJGuoY, et al. (2018) Pl-VIO: Tightly-coupled monocular visual–inertial odometry using point and line features. Sensors18(4): 1159.
14.
HuangG (2019) Visual-inertial navigation: A concise review. In: 2019 IEEE International Conference on Robotics and Automation, Montreal, Canada, 20-24 May, pp. 9572−9582. New York: IEEE.
15.
KongXWuWZhangL, et al. (2015) Tightly-coupled stereo visual-inertial navigation using point and line features. Sensors15(7): 12816−12833.
16.
LeeJParkS (2021) PLF-VINS: Real-time monocular visual-inertial SLAM with point-line fusion and parallel-line fusion. IEEE Robotics and Automation Letters6(4): 7033−7040.
17.
LeuteneggerSLynenSBosseM, et al. (2014) Keyframe-based visual-inertial odometry using nonlinear optimization. International Journal of Robotics Research34(3): 314–334.
18.
LiBFangYZhangX (2016) Visual servo regulation of wheeled mobile robots with an uncalibrated onboard camera. IEEE/ASME Transactions on Mechatronics21(5): 2330−2342.
19.
LiBWangSGeC, et al. (2024) Bi-level intelligent dynamic path planning for an UAV in low-altitude complex urban environment. Transactions of the Institute of Measurement and Control46(1):39-57.
20.
LiCYanFWangS, et al. (2023) A 3D LiDAR odometry for UGVs using coarse-to-fine deep scene flow estimation. Transactions of the Institute of Measurement and Control45(2): 274-286.
21.
LiTPeiLXiangY, et al. (2022) P3-VINS: Tightly-coupled PPP/INS/visual SLAM based on optimization approach. IEEE Robotics and Automation Letters7(3): 7021-7027.
22.
LimHKimYJungK, et al. (2021) Avoiding degeneracy for monocular visual SLAM with point and line features. In: 2021 IEEE International Conference on Robotics and Automation, Xi’an, China, 30 May, pp. 11675−11681. New York: IEEE.
23.
LimHJeonJMyungH (2022) UV-SLAM: Unconstrained line-based SLAM using vanishing points for structural mapping. IEEE Robotics and Automation Letters7(2): 1518−1525.
24.
LiuZShiDLiR, et al. (2021) PLC-VIO: Visual-Inertial Odometry Based on Point-Line Constraints. IEEE Transactions on Automation Science and Engineering19(3): 1880-1897.
25.
LuXYaoyJLiH, et al. (2017) 2-line exhaustive searching for real-time vanishing point estimation in manhattan world. In: 2017 IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, CA, USA, 24-31 March, pp. 345–353. New York: IEEE.
26.
MourikisARoumeliotisS (2007) A multi-state constraint Kalman filter for vision-aided inertial navigation. In: 2007 IEEE International Conference on Robotics and Automation, Roma, Italy, 10-14 April, pp. 3565–3572. New York: IEEE.
27.
QinTLiPShenS (2018) VINS-Mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics34(4): 1004−1020.
28.
QinTPanJCaoS, et al. (2019) A general optimization-based framework for local odometry estimation with multiple sensors. arXiv preprint arXiv 1901.03638.
29.
SchubertDGollTDemmelN, et al. (2018) The TUM VI benchmark for evaluating visual-inertial odometry. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain, 01-05 Oct, pp. 1680-1687. New York: IEEE.
30.
ShenSMichaelNKumarV (2015) Tightly-coupled monocular visual-inertial fusion for autonomous flight of rotorcraft MAVs. In: 2015 IEEE International Conference on Robotics and Automation, Seattle, WA, 26-30 May, pp. 5303–5310. New York: IEEE.
31.
ShiJ (1994) Good features to track. In: 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 21-23 June, pp. 593−600. New York: IEEE.
32.
SuárezIBuenaposadaJBaumelaL (2022) ELSED: Enhanced line SEgment drawing. arXiv 2108.03144.
33.
TomasiCKanadeT (2003) Detection and tracking of point. International Journal of Computer Vision9: 137-154.
34.
Von GioiRJakubowiczJMorelJ, et al. (2010) LSD: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence32(4): 722–732.
35.
WangWChenXLiL, et al. (2023) Cooperative target search and tracking for multi-UAVs based on control barrier functions. Transactions of the Institute of Measurement and Control45(13):2582-2589.
36.
WenHTianJLiD (2020) PLS-VIO: Stereo vision-inertial odometry based on point and line features. In: 2020 International Conference on High Performance Big Data and Intelligent Systems, Shenzhen, China, 23-23 May, pp. 1−7. New York: IEEE.
37.
XuBWangPHeY, et al. (2022) Leveraging structural information to improve point line visual-inertial odometry. IEEE Robotics and Automation Letters7(2): 3483−3490.
38.
ZhangJYangJMaJ. (2024) Monocular visual-inertial odometry leveraging point-line features with structural constraints. The Visual Computer40(2): 647-661.
39.
ZhangLKochR (2013) An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency. Journal of Visual Communication and Image Representation24(7): 794-805.
40.
ZhengFTsaiGZhangZ, et al. (2018) Trifo-VIO: Robust and efficient stereo visual inertial odometry using points and lines. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain, 01-05 October, pp. 3686−3693. New York: IEEE.
41.
ZouDWuYPeiL, et al. (2019) StructVIO: visual-inertial odometry with structural regularity of man-made environments. IEEE Transactions on Robotics35(4): 999−1013.
