IMAGO: Image-guided navigation for visually impaired people

Abstract

Blind and visually impaired persons face many challenges due to isolation, the most important is lacking education due to social immobility. Yet, since the development of the well-known white-red cane, only few advances have been made to increase the mobility of blind people. GPS systems are often used for pedestrian navigation but lack precision, foremost in rural areas where navigation is most often needed. The aim of the IMAGO project is therefore the development of an inexpensive and unobtrusive navigation method for blind and visually impaired persons. Navigation is performed using structure from motion and image-based localization techniques. Route models are created as 3D point clouds through several steps: (i) image acquisition along the routes; (ii) bulk-transfer of images to a server; (iii) feature extraction; (iv) feature matching between images; (v) creation of 3D point cloud with structure from motion. Similarly, the navigation chains seven steps: (a) image acquisition while walking a route; (b) immediate transfer of image to a server; (c) feature extraction; (d) feature matching between image and 3D route model; (e) localization/camera matrix calculation; (f) navigation/calculation of direction based on localization; (g) transfer of direction to user. Current smartphones are used as devices both for recording of routes as well as navigation. Thereby, a high level of dissemination without additional costs is possible, both, within blind people for navigation, as well as seeing people for route creation. Additionally, haptic feedback can be used via a smart cane to reduce auditive feedback. The proposed system yields a high positioning accuracy of 80% of samples being located within 1.6 m. Thus, the system is usable for pedestrian navigation, especially for visually impaired persons.

Keywords

Visually impaired navigation structure from motion image-based localization

Get full access to this article

View all access options for this article.

References

[1]

Agarwal,

Snavely,

Simon,

Seitz and

Szeliski, Building Rome in a day, in: Proc. of the 12th International Conference on Computer Vision, ICCV’09, IEEE Computer Society, Washington, DC, USA, 2009, pp. 72–79.

[2]

Aladrén,

López-Nicolás,

Puig and

J.J.

Guerrero, Navigation assistance for the visually impaired using rgb-d sensor with range expansion, IEEE Systems JournalPP(99) (2014), 1–11.

[3]

Bitsch,

Smith,

Viol and

W.K.

Footpath, Accurate map-based indoor navigation using smartphones, in: Proc. of the 2011 International Conference on Indoor Positioning and Indoor Navigation, 2011.

[4]

Davison,

Reid,

Molton and

Stasse, Monoslam: Real-time single camera slam, IEEE Transactions on Pattern Analysis and Machine Intelligence29(6) (June 2007), 1052–1067, ISSN 0162-8828, doi:10.1109/TPAMI.2007.1049.

[5]

Dellaert,

S.M.

Seitz,

C.E.

Thorpe and

Thrun, Structure from motion without correspondence, in: Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, 2000, pp. 557–564.

[6]

R.W.

Emerson and

Sauerburger, Detecting approaching vehicles at streets with no traffic control, Journal of Visual Impairment and Blindness102(12) (2010), 747–760.

[7]

Fallah,

Apostolopoulos,

Bekris and

Folmer, The user as a sensor: Navigating users with visual impairments in indoor spaces using tactile landmarks, in: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, ACM, New York, NY, USA, 2012, pp. 425–432.

[8]

Filipe,

Fernandes,

Sousa,

Paredes and

Barroso, Blind navigation support system based on Microsoft kinect, in: 4th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-Exclusion, DSAI 2012, Vol. 14, Elsevier, 2012, pp. 94–101.

[9]

M.A.

Fischler and

R.C.

Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM24(6) (1981), 381–395.

10.

[10]

J.-M.

Frahm,

Fite-Georgel,

Gallup,

Johnson,

Raguram,

Wu,

Y.-H.

Jen,

Dunn,

Clipp,

Lazebnik and

Pollefeys, Building Rome on a cloudless day, in: Proc. of the 11th European Conference on Computer Vision: Part IV, ECCV’10, Springer-Verlag, Berlin, Heidelberg, 2010, pp. 368–381.

11.

[11]

F.E.

Grubbs, Procedures for detecting outlying observations in samples, Technometrics11(1) (February 1969), 1–21.

12.

[12]

Huang,

Schmidt and

Gartner (eds), Spatial Knowledge Acquisition in the Context of GPS-Based Pedestrian Navigation, Springer-Verlag, Berlin, Heidelberg, 2012.

13.

[13]

Irschara,

Zach,

Klopschitz and

Bischof, Large-scale, dense city reconstruction from user-contributed photos, Computer Vision and Image Understanding116(1) (Jan. 2012), 2–14, ISSN 1077–3142.

14.

[14]

Karlsson,

Di Bernardo,

Ostrowski,

Goncalves,

Pirjanian and

Munich, The vslam algorithm for robust localization and mapping, in: Robotics and Automation, 2005, pp. 24–29.

15.

[15]

Kef,

Hox and

Habekothé, Social networks of visually impaired and blind adolescents. Structure and effect on well-being, Social Networks22(1) (2000), 73–91.

16.

[16]

E.M.

Knorr and

R.T.

Ng, Algorithms for mining distance-based outliers in large datasets, in: Proc. of the 24rd International Conference on Very Large Data Bases, VLDB ’98, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1998, pp. 392–403.

17.

[17]

Leppäkoski,

Collin and

Takala, Pedestrian navigation based on inertial sensors, indoor map, and wlan signals, Journal of Signal Processing Systems71(3) (2013), 287–296.

18.

[18]

Li and

Becerik-Gerber, Performance-based evaluation of rfid-based indoor location sensing solutions for the built environment, Advanced Engineering Informatics25(3) (2011), 535–546.

19.

[19]

D.G.

Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision60(2) (2004), 91–110.

20.

[20]

Modsching,

Kramer and

ten Hagen, Field trial on GPS accuracy in a medium size city: The influence of built-up, in: Proc. of the 3rd Workshop on Positioning, Navigation and Communication, IEEE, Hannover, Germany, 2006, pp. 209–218.

21.

[21]

Sattler,

Leibe and

Kobbelt, Fast image-based localization using direct 2d-to-3d matching, in: Proc. of the International Conference on Computer Vision, 2011, pp. 667–674.

22.

[22]

Snavely,

S.M.

Seitz and

Szeliski, Photo tourism: Exploring photo collections in 3d, in: Proc. of the ACM SIGGRAPH 2006, ACM, New York, NY, USA, 2006, pp. 835–846.

23.

[23]

Strecha,

Pylvaenaeinen and

Fua, Dynamic and scalable large scale image reconstruction, in: Proc. of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2010, pp. 406–413.

24.

[24]

Streckel and

Koch, Lens model selection for visual tracking, in: Pattern Recognition,

Kropatsch,

Sablatnig and

Hanbury, eds, Lecture Notes in Computer Science, Vol. 3663, Springer, Berlin, Heidelberg, 2005, pp. 41–48, ISBN 978-3-540-28703-2.

25.

[25]

Thrun and

J.J.

Leonard, Simultaneous localization and mapping, in: Springer Handbook of Robotics, Springer-Verlag, Berlin, Heidelberg, 2008, pp. 871–889.

26.

[26]

Untzelmann,

Sattler,

Middelberg and

Kobbelt, A scalable collaborative online system for city reconstruction, in: Proc. of the 2013 IEEE International Conference on Computer Vision Workshops (ICCVW), Dec. 2013, pp. 644–651.

27.

[27]

K.M.

Varpe and

M.P.

Wankhade, Visually impaired assistive system, International Journal of Computer Applications77(16) (September 2013), 5–10.

28.

[28]World Health Organization, Visual impairment and blindness, Fact Sheet 282, World Health Organization, 2014.

29.

[29]

Wu, Towards linear-time incremental structure from motion, in: Proc. of the 2013 International Conference on 3D Vision, 3DV ’13, IEEE Computer Society, Washington, DC, USA, 2013, pp. 127–134.

30.

[30]

Zhang, Iterative point matching for registration of free-form curves and surfaces, International Journal of Computer Vision13(2) (October 1994), 119–152.