While prior studies on visual-to-auditory sensory substitution (VASS) have focused on technological development, practical adoption remains limited. This study explores how students with visual impairment and special educators in South Korea perceive the usability and potential improvements of VASS after brief training. Drawing on semi-structured interviews and thematic analysis, the study identified four overarching themes (i.e., perceived value, influencing factors, real-world challenges, and suggestions for improvement) spanning 18 subthemes. Participants noticed the potential of VASS to enhance learning, spatial reasoning, creativity, and engagement, while also highlighting limitations including high cognitive load, unclear comparative advantage over conventional assistive tools, and the absence of structured training protocols. The findings emphasize the importance of user-centered design, adaptive training, and multisensory integration for future development of sensory substitution systems in educational and daily contexts.
AbboudS.HanassyS.Levy-TzedekS.MaidenbaumS.AmediA. (2014). EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution. Restorative Neurology and Neuroscience, 32(2), 247–257. https://doi.org/10.3233/RNN-130338
2.
ArbelR.HeimlerB.AmediA. (2022). Face shape processing via visual-to-auditory sensory substitution activates regions within the face processing networks in the absence of visual experience. Frontiers in Neuroscience, 16, Article 921321. https://doi.org/10.3389/fnins.2022.921321
3.
Bach-y-RitaP. (1972). Brain mechanisms in sensory substitution. Academic Press.
BraunV.ClarkeV. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.
6.
BrownD. J.ProulxM. J. (2016). Audio–vision substitution for blind individuals: Addressing human information processing capacity limitations. IEEE Journal of Selected Topics in Signal Processing, 10(5), 924–931. https://doi.org/10.1109/JSTSP.2016.2543678
7.
BrownD. J.SimpsonA. J.ProulxM. J. (2014). Visual objects in the auditory system in sensory substitution: How much information do we need?Multisensory Research, 27(5–6), 337–357. https://doi.org/10.1163/22134808-00002462
8.
BrownD. J.SimpsonA. J.ProulxM. J. (2015). Auditory scene analysis and sonified visual images: Does consonance negatively impact on object formation when using complex sonified stimuli?Frontiers in Psychology, 6, Article 1522. https://doi.org/10.3389/fpsyg.2015.01522
9.
BuchsG.MaidenbaumS.Levy-TzedekS.AmediA. (2015). Integration and binding in rehabilitative sensory substitution: Increasing resolution using a new zooming-in approach. Restorative Neurology Andneuroscience, 34(1), 97–105. https://doi.org/10.3233/RNN-150592
10.
ChebatD. R.MaidenbaumS.AmediA. (2015). Navigation using sensory substitution in real and virtual mazes. PLOS ONE, 10(6), Article e0126307. https://doi.org/10.1371/journal.pone.0126307
11.
ChunduryP.ReyazuddinY.JordanJ. B.LazarJ.ElmqvistN. (2023). TactualPlot: Spatializing data as sound using sensory substitution for touchscreen accessibility. IEEE Transactions on Visualization and Computer Graphics, 30(1), 836–846. https://doi.org/10.1109/tvcg.2023.3326937
12.
DavisF. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. https://doi.org/10.2307/249008
13.
Electronics and Telecommunications Research Institute. (2021). Final research report: Analysis of perceptual recognition and resolution of visual-to-auditory conversion signals among individuals with visual impairment.
14.
ElliG. V.BenettiS.CollignonO. (2014). Is there a future for sensory substitution outside academic laboratories?Multisensory Research, 27(5–6), 271–291. https://doi.org/10.1163/22134808-00002460
15.
HaighA.BrownD. J.MeijerP.ProulxM. J. (2013). How well do you see what you hear? The acuity of visual-to-auditory sensory substitution. Frontiers in Psychology, 4, Article 330. https://doi.org/10.3389/fpsyg.2013.00330
16.
Hamilton-FletcherG.ChanK. C. (2021). Auditory scene analysis principles improve image reconstruction abilities of novice vision-to-audio sensory substitution users. In Proceedings of the 43rd annual international conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 5868–5871). IEEE. https://doi.org/10.1109/EMBC46164.2021.9630296
17.
Hamilton-FletcherG.ObristM.WattenP.MengucciM.WardJ. (2016). “I always wanted to see the night sky”: Blind user preferences for sensory substitution devices. In Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 2162–2174). ACM. https://doi.org/10.1145/2858036.2858241
18.
HayesC.ProulxM. J. (2024). Turning a blind eye? Removing barriers to science and mathematics education for students with visual impairments. British Journal of Visual Impairment, 42(2), 544–556. https://doi.org/10.1177/02646196221149561
19.
HeimlerB.Striem-AmitE.AmediA. (2015). Origins of task-specific sensory-independent organization in the visual and auditory brain: Neuroscience evidence, open questions, and clinical implications. Current Opinion in Neurobiology, 35, 169–177. https://doi.org/10.1016/j.conb.2015.09.001
20.
JicolC.Lloyd-EsenkayaT.ProulxM. J.Lange-SmithS.SchellerMO’NeillE.PetriniK. (2020). Efficiency of sensory substitution devices alone and in combination with self-motion for spatial navigation in sighted and visually impaired. Frontiers in Psychology, 11, Article 1443. https://doi.org/10.3389/fpsyg.2020.01443
21.
KristjánssonÁ.MoldoveanuA.JóhannessonÓ. I.BalanO.SpagnolS.ValgeirsdóttirV. V.UnnthorssonR. (2016). Designing sensory-substitution devices: Principles, pitfalls, and potential. Restorative Neurology and Neuroscience, 34(5), 769–787. https://doi.org/10.3233/RNN-160647
22.
Levy-TzedekS.HanassyS.AbboudS.MaidenbaumS.AmediA. (2012). Fast, accurate reaching movements with a visual-to-auditory sensory substitution device. Restorative Neurology and Neuroscience, 30(4), 313–323. https://doi.org/10.3233/RNN-2012-110219
23.
LiangI.SpencerB.SchellerM.ProulxM. J.PetriniK. (2024). Assessing people with visual impairments’ access to information, awareness and satisfaction with high-tech assistive technology. British Journal of Visual Impairment, 42(1), 149–163. https://doi.org/10.1177/02646196221131746
24.
Lloyd-EsenkayaT.Lloyd-EsenkayaVO’NeillE.ProulxM. J. (2020). Multisensory inclusive design with sensory substitution. Cognitive Research: Principles and Implications, 5(1), 37. https://doi.org/10.1186/s41235-020-00240-7
25.
MaidenbaumS.HanassyS.AbboudS.BuchsG.ChebatD. R.Levy-TzedekS.AmediA. (2014). The “EyeCane,” a new electronic travel aid for the blind: Technology, behavior and swift learning. Restorative Neurology and Neuroscience, 32(6), 813–824. https://doi.org/10.3233/RNN-130351
26.
MeijerP. B. L. (1992). An experimental system for auditory image representations. IEEE Transactions on Biomedical Engineering, 39(2), 112–121. https://doi.org/10.1109/10.121642
27.
NetzerY.NovickI.AmediA. (2019). The neural basis of learning visual concepts using sensory substitution. Scientific Reports, 9, Article 13259. https://doi.org/10.1038/s41598-019-49448-0
28.
RicciC.BisioA.RuggeriP.TrombettaL. (2023). Learning spatial representations through audio feedback in immersive environments: Insights for sensory substitution. Frontiers in Psychology, 14, Article 1145673. https://doi.org/10.3389/fpsyg.2023.1145673
29.
RichardsonM.PetriniK.ProulxM. J. (2022). Climb-o-vision: A computer-vision-driven sensory substitution device for rock climbing. In Proceedings of the 2022 CHI conference on human factors in computing systems (pp. 1–6). ACM. https://doi.org/10.1145/3491101.3519680
30.
RogersE. M.SinghalA.QuinlanM. M. (2014). Diffusion of innovations. In StacksD. W.SalwenM. B. (Eds.), An integrated approach to communication theory and research (2nd ed., pp. 432–448). Routledge.
31.
SekiY.SatoT. (2010). A training system of the sensory substitution using virtual environment. Proceedings of the 2010 IEEE Symposium on D User Interfaces, 3, 71–74. https://doi.org/10.1109/3DUI.2010.5444729
32.
Striem-AmitE.GuendelmanM.AmediA. (2012). “Visual” acuity of the congenitally blind using visual-to-auditory sensory substitution. PLOS ONE, 7(3), Article e33136. https://doi.org/10.1371/journal.pone.0033136
33.
WardJ.MeijerP. (2010). Visual experiences in the blind induced by an auditory sensory substitution device. Consciousness and Cognition, 19(1), 492–500. https://doi.org/10.1016/j.concog.2009.10.006
34.
ZhangZ.ThompsonJ. R.ShahA.AgrawalM.SarikayaA.WobbrockJ. O.CutrellE.LeeB. (2024). ChartA11y: Designing accessible touch experiences of visualizations with blind smartphone users. In Proceedings of the 26th international ACM SIGACCESS conference on computers and accessibility (pp. 1–15). ACM.
35.
ZhaoY.NacentaM. A.SukhaiM. A.SomanathS. (2024). TADA: Making node-link diagrams accessible to blind and low-vision people. In Proceedings of the 2024 CHI conference on human factors in computing systems (pp. 1–20). ACM.
