Visual-to-auditory sensory substitution with passive movements in a double participant setup

Abstract

In the field of sensory substitution, research has highlighted the role of participants’ actions on the sensors of sensory substitution devices. These observations are in line with the conception of perception as a dynamic process in which action plays an essential role. However, a debate remains between several conceptions. According to the ecological psychology approach, action may correspond to voluntary movements, but also to passive movements that expose us to invariants and enable perception. For the enactive cognitive science approach, action corresponds mainly to voluntary movements, the aim of which is to test sensorimotor contingencies, and give rise to perception. To contribute to this debate, we have set up a visual-to-auditory sensory substitution device coupled with a pantograph system for transferring identical movements. This makes it possible to test two participants simultaneously, one acting voluntarily on the device’s sensors, the other subjected to passive movements that are nonetheless correctly associated with auditory feedback. Participants were asked to recognize 2D shapes, and our results show that they improved their perception irrespective of whether the experimental condition was active or passive. Thus, our results confirm that sensory substitution is possible via passive movements.