The present study investigates human multisensory perception of sound and vibration, highlighting its potential impact in the design of novel user interfaces, including those used in the automobile industry. Specifically, the present study investigates whether front—back sound localization could be altered by concurrent whole-body vibration. Previous research has shown that, when auditory and tactile stimuli are presented synchronously but from different positions, the perceived location of the auditory event is mislocalized to the location of the tactile stimulus. Here, sounds were presented at the front or the back of participants, in isolation, or together with vibrations. Participants made a three-alternative forced choice regarding their perceived location of the sounds. Results indicate that front—back sound localization was affected by the presence of concurrent vibrations, which biased the localization of front sounds towards the participants' rear space. Since the perceived location of events modulates the perceivers' understanding and involvement in these events, the possibility of manipulating the location of sound events using vibrations has a potential for the design of multisensory interfaces such as those included in automotive applications, where it is strongly needed to capture the attention of drivers, to provide navigational information, and to reduce sensory load.
SpenceC.HoC.‘Tactile and multisensory spatial warning signals for drivers.’IEEE Trans. Haptics1 (2) (2008): 121–129.
2.
SköldA.Integrative analyses of perception and reaction to information and warning sounds in vehicles Doctoral dissertation, Chalmers University of Technology, Gothenburg, Sweden, 2008.
3.
BegaultD. R.‘Head-up auditory displays for traffic collision avoidance system advisories: A preliminary investigation.’Hum. Factors35 (1993): 707–717.
4.
BellottiF.BertaR.De GloriaA.MargaroneM.‘Using 3D sound to improve the effectiveness of the advanced driver assistance systems.’J. Personal Ubiquitous Comput.6 (2002): 155–163.
5.
RochlisJ. L.NewmanD. J.‘A tactile display for international space station (ISS) extravehicular activity (EVA).’Aviation, Space, Environ. Med.71 (2000): 571–578.
6.
van ErpJ. B. F.van VeenH. A.H.C.‘Vibrotactile in-vehicle navigation system.’Transpn Res., Part F: Traffic Psychol. Behav.7 (2004): 247–256.
7.
van ErpJ. B. F.JansenC.DobbinsT.van VeenH. A. H. C.‘Vibrotactile waypoint navigation at sea and in the air: Two case studies.’ In Proceedings of EuroHaptics, Munich, Germany, 5–7 June 2004, pp. 166–173 (Herbert Hieronymus, Munich).
VäljamäeA.Tajadura-JiménezA.LarssonP.VästfjällD.KleinerM.‘Handheld experiences: Using audio to enhance the illusion of self-motion.’IEEE MultiMedia15 (2008): 68–75.
KitagawaN.IchiharaS.‘Hearing visual motion in depth.’Nature416 (6877) (2002): 172–174.
12.
WitkinH. A.WapnerS.LeventhalT.‘Sound localization with conflicting visual and auditory cues.’J. Expl Psychol.43 (1952): 58–67.
13.
VroomenJ.De GelderB.‘Perceptual effects of crossmodal stimulation: The cases of ventriloquism and the freezing phenomenon.’ In The handbook of multisensory processes (Eds CalvertG.SpenceC.SteinB. E.), 2005, pp. 141–150 (MIT Press, Cambridge, Massachusetts).
14.
SpenceC.DriverJ. Ed. Crossmodal space and crossmodal attention, (Oxford University Press, Oxford2004).
15.
CalvertG.SpenceC.SteinB. E.The handbook of multisensory processes, (MIT Press, Cambridge, Massachusetts2004).
WelchR. B.WarrenD. H.‘Intersensory interactions.’ In Handbook of perception and performance: Sensory processes and perception, vol. 1 (Eds BoffK. R.KaufmanL.ThomasJ. P.), 1986, pp. 25-21–25-36 (John Wiley, New York).
19.
MillerJ.‘Discrete versus continuous stage models of human information-processing: In search of partial output.’J. Expl Psychol. — Hum. Perception Performance8 (2) (1982): 273–296.
20.
Tajadura-JiménezA.KitagawaN.VäljamäeA.ZampiniM.MurrayM. M.SpenceC.‘Auditory—somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra.’Neuropsychologia47 (1) (2009): 195–203.
21.
BegaultD. R.3-D sound for virtual reality and multimedia, (Academic Press, San Diego, California1994).
22.
CaclinA.Soto-FaracoS.KingstoneA.SpenceC.‘Tactile ‘capture’ of audition.’Perception Psychophys.64 (4) (2002): 616–630.
23.
Tajadura-JiménezA.VäljamäeA.VästfjällD.‘Self-representation in mediated environments: The experience of emotions modulated by auditory—vibrotactile heartbeat.’CyberPsychol. Behavior11 (2008): 33–38.
24.
David McAlpineA.JiangD.PalmerA. R.‘A neural code for low-frequency sound localization in mammals.’Nature Neuroscience4 (2001): 396–401.
25.
VäljamäeA.LarssonP.VästfjällD.KleinerM.‘Vibrotactile enhancement of auditory-induced self-motion and spatial presence.’J. Audio Engng Soc.54 (10) (2006): 954–963.
26.
LewkowiczD. J.‘The development of temporal and spatial intermodal perception.’ In Cognitive contributions to the perception of spatial and temporal events (Ed. AscherslebenG.), 1999, pp. 395–420 (Elsevier, Amsterdam).
27.
Neurobehavioral Systems, Auditory, visual and multi-modal stimulus delivery for neuroscience, available from http://www.neurobs.com/presentation.
28.
Tajadura-JiménezA.LarssonP.VäljamäeA.VästfjällD.KleinerM.‘When room size matters: Acoustic influences on emotional responses to sounds.’Emotion10 (3) (2010): 416–422.
29.
ÖhmanA.‘Fear and anxiety as emotional phenomena: Clinical phenomenology, evolutionary perspectives, and information-processing mechanisms.’ In Handbook of emotions (Eds LewisM.HavilandJ. M.), 1993, pp. 511–536 (Guilford Press, New York).
30.
PhelpsE. A.LeDouxJ. E.‘Contributions of the amygdala to emotion processing: From animal models to human behavior.’Neuron48 (2005): 175–187.
31.
PhelpsE. A.LingS.CarrascoM.‘Emotion facilitates perception and potentiates the perceptual benefits of attention.’Psychol. Sci.17 (2006): 292–299.
32.
Tajadura-JiménezA.Embodied psychoacoustics: Spatial and multisensory determinants of auditory-induced emotion Doctoral dissertation, Chalmers University of Technology, Gothenburg, Sweden, 2008.
33.
LarssonP.Virtually hearing, seeing, and being: Room acoustics, presence, and audiovisual environments Doctoral dissertation, Chalmers University of Technology, Gothenburg, Sweden, 2005.
34.
Tajadura-JiménezA.VäljamäeA.KitagawaN.HoH.‘Whole-body vibration influences sound localization in the median plane.’ In Proceedings of the Tenth Annual International Workshop on Presence (PRESENCE 2007), Barcelona, Spain, 25–27 October 2007, pp. 35–39 (International Society for Presence Research, Philadelphia, Pennsylvania).
