Sage Journals: Discover world-class research

Abstract

In the field of sensory substitution, research has highlighted the role of participants’ actions on the sensors of sensory substitution devices. These observations are in line with the conception of perception as a dynamic process in which action plays an essential role. However, a debate remains between several conceptions. According to the ecological psychology approach, action may correspond to voluntary movements, but also to passive movements that expose us to invariants and enable perception. For the enactive cognitive science approach, action corresponds mainly to voluntary movements, the aim of which is to test sensorimotor contingencies, and give rise to perception. To contribute to this debate, we have set up a visual-to-auditory sensory substitution device coupled with a pantograph system for transferring identical movements. This makes it possible to test two participants simultaneously, one acting voluntarily on the device’s sensors, the other subjected to passive movements that are nonetheless correctly associated with auditory feedback. Participants were asked to recognize 2D shapes, and our results show that they improved their perception irrespective of whether the experimental condition was active or passive. Thus, our results confirm that sensory substitution is possible via passive movements.

Keywords

Sensory substitution perceptual supplementation perception action ecological psychology passive movement

Get full access to this article

View all access options for this article.

References

Adolph

K. E.

Kretch

K. S.

(2015). Gibson’s theory of perceptual learning. In Wright

J. D.

(Éd.), International encyclopedia of the social & behavioral sciences (2nd ed., pp. 127–134). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.23096-1

Arnold

Pesnot-Lerousseau

Auvray

(2017). Individual differences in sensory substitution. Multisensory Research, 30(6), 579–600. https://doi.org/10.1163/22134808-00002561

Auvray

(2019). Multisensory and spatial processes in sensory substitution. Restorative Neurology and Neuroscience, 37(6), 609–619. https://doi.org/10.3233/RNN-190950

Auvray

Hanneton

Lenay

O’Regan

(2005). There is something out there: Distal attribution in sensory substitution, twenty years later. Journal of Integrative Neuroscience, 4(4), 505–521.

Bach-y-Rita

Collins

C. C.

Saunders

F. A.

White

Scadden

(1969). Vision substitution by tactile image projection. Nature, 221(5184), 963–964. https://doi.org/10.1038/221963a0

Bermejo

Di Paolo

E. A.

Hüg

M. X.

Arias

(2015). Sensorimotor strategies for recognizing geometrical shapes: A comparative study with different sensory substitution devices. Frontiers in Psychology, 6, 679. https://doi.org/10.3389/fpsyg.2015.00679

Díaz

Barrientos

Jacobs

D. M.

Travieso

. (2012). Action-contingent vibrotactile flow facilitates the detection of ground level obstacles with a partly virtual sensory substitution device. Human Movement Science, 31(6), 1571–1584. https://doi.org/10.1016/j.humov.2012.05.006

Elkin

L. A.

Kay

Higgins

J. J.

Wobbrock

J. O.

(2021). An aligned rank transform procedure for multifactor contrast tests. The 34th Annual ACM Symposium on User Interface Software and Technology (UIST ‘21), (pp. 754–768). Virtual Event, USA. https://doi.org/10.1145/3472749.3474784

Favela

L. H.

Riley

M. A.

Shockley

Chemero

(2018). Perceptually equivalent judgements made visually and via haptic sensory-substitution devices. Ecological Psychology 30, 326–345. https://doi.org/10.1080/10407413.2018.1473712

10.

Froese

Ortiz-Garin

G. U.

(2020). Where is the action in perception? An exploratory study with a haptic sensory substitution device. Frontiers in Psychology, 11, 809. https://doi.org/10.3389/fpsyg.2020.00809

11.

Froese

McGann

Bigge

Spiers

Seth

A. K.

(2012). The Enactive Torch: A new tool for the science of perception. IEEE Transactions on Haptics, 5(4), 365–375. https://doi.org/10.1109/TOH.2011.57

12.

Gapenne

Ammar

A. A.

Lenay

Rovira

(2003). TACTOS: A special computer interface for the reading and writing of 2D forms in blind people. Human-Computer Interaction: Universal Access in HCI: Inclusive Design in the Information Society, Proceedings of HCI International 2003 (the 10th International Conference on Human-Computer Interaction), HCI 2003, Crete, Greece, June 22–27, 2003 (Vol. 4, pp. 1270–1274).

13.

Gibson

J. J.

(1979). The ecological approach to visual perception: Classic edition. Houghton Mifflin.

14.

Haggard

Chambon

(2012). Sense of agency. Current Biology, 22(10), R390–R392. https://doi.org/10.1016/j.cub.2012.02.040

15.

Hanneton

Gapenne

Genouel

Lenay

Marque

(1999). Dynamics of shape recognition through a minimal visuo-tactile sensory substitution interface. In Proc., Third International Conference on Cognitive and Neural Systems, Boston, MA. https://hal.science/hal-00185599

16.

Heft

(2020). Ecological psychology and enaction theory: Divergent groundings. Frontiers in Psychology, 11, 991. https://www.frontiersin.org/articles/10.3389/fpsyg.2020.00991

17.

Held

Hein

(1963). Movement-produced stimulation in the development of visually guided behavior. Journal of Comparative and Physiological Psychology, 56, 872–876. https://doi.org/10.1037/h0040546

18.

Heller

M. A.

(1986). Active and passive tactile braille recognition. Bulletin of the Psychonomic Society, 24(3), 201–202. https://doi.org/10.3758/BF03330548

19.

Israël

Grasso

Georges-Francois

Tsuzuku

Berthoz

(1997). Spatial memory and path integration studied by self-driven passive linear displacement. I. Basic properties. Journal of Neurophysiology, 77(6), 3180–3192. https://doi.org/10.1152/jn.1997.77.6.3180

20.

Lenay

Gapenne

Hanneton

Marque

Genouëlle

(2003). Sensory substitution: Limits and perspectives. In Hatwell

Streri

Gentaz

(Eds.), Touching for knowing: Cognitive psychology of haptic manual perception (pp. 276–292). John Benjamins Publishing Company. https://doi.org/10.1075/aicr.53.22len

21.

Lobo

(2019). Current alternatives on perceptual learning: Introduction to special issue on post-cognitivist approaches to perceptual learning. Adaptive Behavior, 27(6), 355–362. https://doi.org/10.1177/1059712319875147

22.

Maturana

H. R.

Varela

F. J.

(1987). The tree of knowledge: The biological roots of human understanding (p. 263). New Science Library/Shambhala Publications.

23.

Meijer

P. B.

(1992). An experimental system for auditory image representations. IEEE Transactions on Bio-Medical Engineering, 39(2), 112–121. https://doi.org/10.1109/10.121642

24.

Norimoto

Ikegaya

(2015). Visual cortical prosthesis with a geomagnetic compass restores spatial navigation in blind rats. Current Biology, 25(8), 1091–1095. https://doi.org/10.1016/j.cub.2015.02.063

25.

O’Regan

J. K.

Noë

(2001). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 939–973; discussion 973–1031. https://doi.org/10.1017/s0140525x01000115

26.

Paolo

E. D.

Buhrmann

Barandiaran

(2017). Sensorimotor life: An enactive proposal. Oxford University Press.

27.

Perrotta

M. V.

Asgeirsdottir

Eagleman

D. M.

(2021). Deciphering sounds through patterns of vibration on the skin. Neuroscience, 458, 77–86. https://doi.org/10.1016/j.neuroscience.2021.01.008

28.

Rizzolatti

Craighero

(2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169–192. https://doi.org/10.1146/annurev.neuro.27.070203.144230

29.

Rovira

Gapenne

(2009). Analytical and holistic strategies employed by blind adolescents in task of tactile classification on traditional and computerized media. Journal of Visual Impairment and Blindness, 103(7), 430–435.

30.

Roy

Menant

Hacquemand

Vallier

Ensch

Rovira

Gapenne

(2019). A case of visuo-auditory sensory substitution in rats. Behavioural Processes, 164, 157–166. https://doi.org/10.1016/j.beproc.2019.05.001

31.

Sangati

Lobo

Estelle

Sangati

Tavassoli

Froese

(2023). Uncovering the role of intention in active and passive perception. Proceedings of the Annual Meeting of the Cognitive Science Society, 45, 663–670. https://escholarship.org/uc/item/6wj762mq

32.

Siegle

J. H.

Warren

W. H

. (2010). Distal Attribution and Distance Perception in Sensory Substitution. Perception, 39(2), 208–223. https://doi.org/10.1068/p6366

33.

Smith

A. M.

Chapman

C. E.

Donati

Fortier-Poisson

Hayward

(2009). Perception of simulated local shapes using active and passive touch. Journal of Neurophysiology, 102(6), 3519–3529. https://doi.org/10.1152/jn.00043.2009

34.

Sribunruangrit

Marque

Lenay

Gapenne

Vanhoutte

(2002). Braille box: Analysis of the parallelism concept to access graphic information for blind people. Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society, Engineering in Medicine and Biology, 3, 2424–2425. https://doi.org/10.1109/IEMBS.2002.1053356

35.

Stillman

B. C.

(2002). Making sense of proprioception: The meaning of proprioception, kinaesthesia and related terms. Physiotherapy, 88(11), 667–676. https://doi.org/10.1016/S0031-9406(05)60109-5

36.

Strelow

E. R.

Warren

D. H.

Sonnier

B. J.

Riesen

A. H.

Kay

Sinton

(1987). Behavioral observations of sensory substitution in neonate macaques (Macaca arctoides). Behavioral Neuroscience, 101(5), 738–741.

37.

Thomson

E. E.

Carra

Nicolelis

M. A. L.

(2013). Perceiving invisible light through a somatosensory cortical prosthesis. Nature Communications, 4, 1482. https://doi.org/10.1038/ncomms2497

38.

Tyler

Danilov

Bach-y-Rita

(2003). Closing an open-loop control system: Vestibular substitution through the tongue. Journal of Integrative Neuroscience, 2(2), 159–164. https://doi.org/10.1142/s0219635203000263

39.

Van Doorn

G. H.

Dubaj

Wuillemin

D. B.

Richardson

B. L.

Symmons

M. A

. (2012). Cognitive Load Can Explain Differences in Active and Passive Touch. In Isokoski

Springare

(Eds.), Haptics: Perception, Devices, Mobility, and Communication (pp. 91–102). Springer. https://doi.org/10.1007/978-3-642-31401-8_9

40.

Van Doorn

G. H.

Symmons

M. A

. (2013). Biases in visuo-haptic matching of curvature. 2013 World Haptics Conference (WHC) (pp. 549–553). Daejeon, South Korea. https://doi.org/10.1109/WHC.2013.6548467

41.

Walk

R. D.

Shepherd

J. D.

Miller

D. R.

(1988). Attention and the depth perception of kittens. Bulletin of the Psychonomic Society, 26(3), 248–251. https://doi.org/10.3758/BF03337301

42.

Ward

Wright

(2014). Sensory substitution as an artificially acquired synaesthesia. Neuroscience & Biobehavioral Reviews, 41, 26–35. https://doi.org/10.1016/j.neubiorev.2012.07.007

43.

White

B. W.

Saunders

F. A.

Scadden

Bach-y-Rita

Collins

C. C.

(1970). Seeing with the skin. Perception & Psychophysics, 7(1), 23–27. https://doi.org/10.3758/BF03210126

44.

Wobbrock

J. O.

Findlater

Gergle

Higgins

J. J.

(2011). The aligned rank transform for nonparametric factorial analyses using only ANOVA procedures. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 143–146). Vancouver, BC, Canada. https://doi.org/10.1145/1978942.1978963

45.

Ziat

Gapenne

Stewart

Lenay

(2007). Haptic recognition of shapes at different scales: A comparison of two methods of interaction. Interacting with Computers, 19(1), 121–132. https://doi.org/10.1016/j.intcom.2006.07.004

Visual-to-auditory sensory substitution with passive movements in a double participant setup

Abstract

Keywords

Get full access to this article

References