Previous research has shown that task-irrelevant auditory/haptic input semantically congruent with a target visual object facilitates visual search, indicating that cross-modal congruency influences goal-directed attentional control. The present study examined whether haptic input involuntarily shifts spatial attention to the congruent visual object even though it was not a search target. Participants identified the arrow direction presented above or below a central gaze fixation point while clasping a specifically shaped item in their hand. Two task-irrelevant pictures with specific shapes preceded the arrow. Results showed a significant interaction between visual and haptic shapes: Participants responded faster when the visual object shared the shape of the item clasped in their hand than when the two shapes differed, indicating that haptic–visual shape congruency modulates spatial attention. Thus, cross-modal congruency can affect involuntary attentional orienting as well as goal-directed attentional control.
ChenY. C.SpenceC. (2010). When hearing the bark helps to identify the dog: Semantically-congruent sounds modulate the identification of masked pictures. Cognition, 114(3), 389–404.https://doi.org/10.1016/j.cognition.2009.10.012
3.
ChenY. C.SpenceC. (2011). Crossmodal semantic priming by naturalistic sounds and spoken words enhances visual sensitivity. Journal of Experimental Psychology: Human Perception and Performance, 37(5), 1554–1568.https://doi.org/10.1037/a0024329
4.
ChiouR.RichA. N. (2012). Cross-modality correspondence between pitch and spatial location modulates attentional orienting. Perception, 41(3), 339–353. https://doi.org/10.1068/p7161
DriverJ.SpenceC. (1998b). Cross–modal links in spatial attention. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 353(1373), 1319–1331.https://doi.org/10.1098/rstb.1998.0286
7.
FaulF.ErdfelderE.BuchnerA.LangA. G. (2009). Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.https://doi.org/10.3758/BRM.41.4.1149
8.
IordanescuL.GraboweckyM.FranconeriS.TheeuwesJ.SuzukiS. (2010). Characteristic sounds make you look at target objects more quickly. Attention, Perception & Psychophysics, 72(7), 1736–1741.https://doi.org/10.3758/APP.72.7.1736
9.
IordanescuL.GraboweckyM.SuzukiS. (2011). Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets. Acta Psychologica, 137(2), 252–259.https://doi.org/10.1016/j.actpsy.2010.07.017
KennettS.EimerM.SpenceC.DriverJ. (2001). Tactile-visual links in exogenous spatial attention under different postures: Convergent evidence from psychophysics and ERPs. Journal of Cognitive Neuroscience, 13(4), 462–478.https://doi.org/10.1162/08989290152001899
12.
KvasovaD.Garcia-VernetL.Soto-FaracoS. (2019). Characteristic sounds facilitate object search in real-life scenes. Frontiers in Psychology, 10, 2511. https://doi.org/10.3389/fpsyg.2019.02511
MaezawaT.KiyosawaM.KawaharaJ. I. (2022). Auditory enhancement of visual searches for event scenes. Attention, Perception, & Psychophysics, 84(2), 427–441.https://doi.org/10.3758/s13414-021-02433-8
15.
MolholmS.RitterW.JavittD. C.FoxeJ. J. (2004). Multisensory visual–auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex, 14(4), 452–465.https://doi.org/10.1093/cercor/bhh007
16.
PelliD. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. https://doi.org/10.1163/156856897x00366
17.
PesquitaA.BrennanA. A.EnnsJ. T.Soto-FaracoS. (2013). Isolating shape from semantics in haptic-visual priming. Experimental Brain Research, 227(3), 311–322.https://doi.org/10.1007/s00221-013-3489-1
RealesJ. M.BallesterosS. (1999). Implicit and explicit memory for visual and haptic objects: Cross-modal priming depends on structural descriptions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25(3), 644–663.https://doi.org/10.1037/0278-7393.25.3.644
SpenceC.NichollsM. E.GillespieN.DriverJ. (1998). Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Perception & Psychophysics, 60(4), 544–557.https://doi.org/10.3758/BF03206045
23.
Van der BurgE.OliversC. N.BronkhorstA. W.TheeuwesJ. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34(5), 1053–1065.https://doi.org/10.1037/0096-1523.34.5.1053
24.
WilliamsJ. R.MarkovY. A.TiurinaN. A.StörmerV. S. (2022). What you see is what you hear: Sounds alter the contents of visual perception. Psychological Science, 33(12), 2109–2122.https://doi.org/10.1177/09567976221121348
25.
WilliamsJ. R.StörmerV. S. (2024). Cutting through the noise: Auditory scenes and their effects on visual object processing. Psychological Science, 35(7), 814–824.https://doi.org/10.1177/09567976241237737
