The sound-induced flash illusion (SIFI) is a classic phenomenon in audiovisual integration research, reflecting the interactive mechanisms in multimodal information processing. This study aimed to recruit 30 healthy participants to explore the effects of sound and flash presentation frequencies and spatial location of stimuli on the perceptual decision-making. Previous studies have used illusion rate or accuracy as the core measurement indices, which fail to reflect the underlying cognitive processes of perceptual decision-making and thus have certain limitations. Therefore, this study aims to model the process of behavioral responses in the task. Among the perceptual decision-making models—Only the Hierarchical Drift Diffusion Model (HDDM) is suitable for the experimental task in this study. We used the drift rate, which reflects the speed of perceptual information accumulation in the model, as the core indicator. The results showed that both the sound-flash combinations and spatial location information significantly influenced evidence accumulation. Furthermore, the overall drift rate for right-side spatial presentation conditions was higher than for left-side conditions. The left-side advantage in B1F1 may reflect the right hemisphere's superiority in spatial attention, while the right-side advantage in B2F1 may reflect hemispheric differences in audiovisual integration. This study is the first to introduce HDDM in analyzing the spatial modulation mechanism within the SIFI paradigm, confirming that spatial location information not only has an independent main effect but can also interact with sensory combination factors to dynamically influence perceptual decision-making.
ApthorpD.AlaisD.BoenkeL. T. (2013). Flash illusions induced by visual, auditory, and audiovisual stimuli. Journal of Vision, 13(5), 3–3. https://doi.org/10.1167/13.5.3
2.
BidelmanG. M.HeathS. T. (2019). Enhanced temporal binding of audiovisual information in the bilingual brain. Bilingualism: Language and Cognition, 22(4), 752–762. https://doi.org/10.1017/S1366728918000408
3.
BizleyJ. K.Shinn-CunninghamB. G.LeeA. K. C. (2012). Nothing is irrelevant in a noisy world: Sensory illusions reveal obligatory within-and across-modality integration. Journal of Neuroscience, 32(39), 13402–13410. https://doi.org/10.1523/JNEUROSCI.2495-12.2012
4.
CarlandM. A.ThuraD.CisekP. (2019). The urge to decide and act: Implications for brain function and dysfunction. The Neuroscientist, 25(5), 491–511. https://doi.org/10.1177/1073858419841553
5.
ChanJ. S.ConnollyS. K.SettiA. (2018). The number of stimulus-onset asynchronies affects the perception of the sound-induced flash illusion in young and older adults. Multisensory Research, 31(3-4), 175–190. https://doi.org/10.1163/22134808-00002605
6.
ChandrasekaranC. (2017). Computational principles and models of multisensory integration. Current Opinion in Neurobiology, 43, 25–34. https://doi.org/10.1016/j.conb.2016.11.002
7.
ChangC.WangE.YangJ.LuanX.WangA.ZhangM. (2023). Differences in eccentricity for sound-induced flash illusion in four visual fields. Perception, 52(1), 56–73. https://doi.org/10.1177/03010066221136670
ChenY.-C.MaurerD.LewisT. L.SpenceC.ShoreD. I. (2017). Central–peripheral differences in audiovisual and visuotactile event perception. Attention, Perception, & Psychophysics, 79(8), 2552–2563. https://doi.org/10.3758/s13414-017-1396-4
10.
CohenJ. (2009). Statistical power analysis for the behavioral sciences (2. ed., reprint). Psychology Press.
11.
CorbettaM.ShulmanG. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3(3), 201–215. https://doi.org/10.1038/nrn755
DiederichA. (2008). A further test of sequential-sampling models that account for payoff effects on response bias in perceptual decision tasks. Perception and Psychophysics, 70(2), 229–256. https://doi.org/10.3758/PP.70.2.229
14.
DowningC. J. (1988). Expectancy and visual-spatial attention: Effects on perceptual quality. Journal of Experimental Psychology: Human Perception and Performance, 14(2), 188. https://doi.org/10.1037/0096-1523.14.2.188
15.
DrugowitschJ.DeAngelisG. C.AngelakiD. E.PougetA. (2015). Tuning the speed-accuracy trade-off to maximize reward rate in multisensory decision-making. Elife, 4, e06678. https://doi.org/10.7554/eLife.06678
16.
DrugowitschJ.DeAngelisG. C.KlierE. M.AngelakiD. E.PougetA. (2014). Optimal multisensory decision-making in a reaction-time task. Elife, 3, e03005. https://doi.org/10.7554/eLife.03005
17.
FalchierA.ClavagnierS.BaroneP.KennedyH. (2002). Anatomical evidence of multimodal integration in primate striate cortex. Journal of Neuroscience, 22(13), 5749–5759. https://doi.org/10.1523/JNEUROSCI.22-13-05749.2002
18.
FaulF.ErdfelderE.LangA.-G.BuchnerA. (2007). G*power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/bf03193146
19.
FetschC. R.NoppeneyU. (2023). How the brain controls decision making in a multisensory world. Philosophical Transactions of the Royal Society B: Biological Sciences, 378, 1886. https://doi.org/10.1098/rstb.2022.0332
20.
ForstmannB. U.RatcliffR.WagenmakersE.-J. (2016). Sequential sampling models in cognitive neuroscience: Advantages, applications, and extensions. Annual Review of Psychology, 67(1), 641–666. https://doi.org/10.1146/annurev-psych-122414-033645
21.
Foss-FeigJ. H.KwakyeL. D.CascioC. J.BurnetteC. P.KadivarH.StoneW. L.WallaceM. T. (2010). An extended multisensory temporal binding window in autism spectrum disorders. Experimental Brain Research, 203(2), 381–389. https://doi.org/10.1007/s00221-010-2240-4
22.
GavinN.HirstR. J.McGovernD. P. (2022). The magnitude of the sound-induced flash illusion does not increase monotonically as a function of visual stimulus eccentricity. Attention, Perception, & Psychophysics, 84(5), 1689–1698. https://doi.org/10.3758/s13414-022-02493-4
23.
GingrasG.RowlandB. A.SteinB. E. (2009). The differing impact of multisensory and unisensory integration on behavior. Journal of Neuroscience, 29(15), 4897–4902. https://doi.org/10.1523/JNEUROSCI.4120-08.2009
24.
HarrisC. R.MillmanK. J.Van Der WaltS. J.GommersR.VirtanenP.CournapeauD.WieserE.TaylorJ.BergS.SmithN. J.KernR.PicusM.HoyerS.Van KerkwijkM. H.BrettM.HaldaneA.Del RíoJ. F.WiebeM.PetersonP.OliphantT. E. (2020). Array programming with NumPy. Nature, 585(7825), 357–362. https://doi.org/10.1038/s41586-020-2649-2
25.
HaßK.SinkeC.ReeseT.RoyM.WiswedeD.DilloW.OranjeB.SzycikG. R. (2017). Enlarged temporal integration window in schizophrenia indicated by the double-flash illusion. Cognitive Neuropsychiatry, 22(2), 145–158. https://doi.org/10.1080/13546805.2017.1287693
26.
HirstR. J.McGovernD. P.SettiA.ShamsL.NewellF. N. (2020). What you see is what you hear: Twenty years of research using the sound-induced flash illusion. Neuroscience & Biobehavioral Reviews, 118, 759–774. https://doi.org/10.1016/j.neubiorev.2020.09.006
ItoY.MatsumotoH.KobayasiK. I. (2023). Subjective audibility modulates the susceptibility to sound-induced flash illusion: Effect of loudness and auditory masking. Multisensory Research, 36, 613–629. https://doi.org/10.1163/22134808-bja10109
29.
JeffreysH. (1998). The theory of probability. OuP Oxford.
KeilJ.MüllerN.HartmannT.WeiszN. (2014). Prestimulus beta power and phase synchrony influence the sound-induced flash illusion. Cerebral Cortex, 24(5), 1278–1288. https://doi.org/10.1093/cercor/bhs409
32.
KleinerM.BrainardD.PelliD. (2007). What’s new in psychtoolbox-3?
33.
KumpikD. P.RobertsH. E.KingA. J.BizleyJ. K. (2014). Visual sensitivity is a stronger determinant of illusory processes than auditory cue parameters in the sound-induced flash illusion. Journal of Vision, 14(7), 12–12. https://doi.org/10.1167/14.7.12
34.
LeeH.NoppeneyU. (2014). Music expertise shapes audiovisual temporal integration windows for speech, sinewave speech, and music. Frontiers in Psychology, 5, 868. https://doi.org/10.3389/fpsyg.2014.00868
35.
LiesefeldH. R.JanczykM. (2019). Combining speed and accuracy to control for speed-accuracy trade-offs(?). Behavior Research Methods, 51(1), 40–60. https://doi.org/10.3758/s13428-018-1076-x
MishraJ.MartinezA.SejnowskiT. J.HillyardS. A. (2007). Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. Journal of Neuroscience, 27(15), 4120–4131. https://doi.org/10.1523/JNEUROSCI.4912-06.2007
41.
MurrayC. A.De Larrea-ManceraE. S. L.GlicksohnA.ShamsL.SeitzA. R. (2020). Revealing multisensory benefit with diffusion modeling. Journal of Mathematical Psychology, 99, 102449. https://doi.org/10.1016/j.jmp.2020.102449
42.
NarinesinghC.GoltzH. C.WongA. M. F. (2017). Temporal binding window of the sound-induced flash illusion in amblyopia. Investigative Opthalmology & Visual Science, 58(3), 1442. https://doi.org/10.1167/iovs.16-21258
43.
OdegaardB.WoznyD. R.ShamsL. (2015). Biases in visual, auditory, and audiovisual perception of space. PLOS Computational Biology, 11(12), e1004649. https://doi.org/10.1371/journal.pcbi.1004649
44.
PanW.GengH.ZhangL.FenglerA.FrankM. J.ZhangR.-Y.Chuan-PengH. (2025). dockerHDDM: A user-friendly environment for Bayesian hierarchical drift-diffusion modeling. Advances in Methods and Practices in Psychological Science, 8(1), 25152459241298700. https://doi.org/10.1177/25152459241298700
45.
PowersA. R.HillockA. R.WallaceM. T. (2009). Perceptual training narrows the temporal window of multisensory binding. Journal of Neuroscience, 29(39), 12265–12274. https://doi.org/10.1523/JNEUROSCI.3501-09.2009
RocklandK. S.OjimaH. (2003). Multisensory convergence in calcarine visual areas in macaque monkey. International Journal of Psychophysiology, 50(1-2), 19–26. https://doi.org/10.1016/S0167-8760(03)00121-1
ShamsL.IwakiS.ChawlaA.BhattacharyaJ. (2005). Early modulation of visual cortex by sound: An MEG study. Neuroscience Letters, 378(2), 76–81. https://doi.org/10.1016/j.neulet.2004.12.035
ShulmanG. L.PopeD. L. W.AstafievS. V.McAvoyM. P.SnyderA. Z.CorbettaM. (2010). Right hemisphere dominance during spatial selective attention and target detection occurs outside the dorsal frontoparietal network. Journal of Neuroscience, 30(10), 3640–3651. https://doi.org/10.1523/JNEUROSCI.4085-09.2010
53.
SunY.ZhouH.LiuC.WangA.YueC.ZhangM. (2022). Sound-induced flash illusion is modulated by the depth of auditory stimuli: Evidence from younger and older adults. Attention, Perception, & Psychophysics, 84(6), 2040–2050. https://doi.org/10.3758/s13414-022-02537-9
54.
TremblayC.ChampouxF.VossP.BaconB. A.LeporeF.ThéoretH. (2007). Speech and non-speech audio-visual illusions: A developmental study. PLOS One, 2(8), e742. https://doi.org/10.1371/journal.pone.0000742
WieckiT. V.SoferI.FrankM. J. (2013). HDDM: Hierarchical Bayesian estimation of the drift-diffusion model in python. Frontiers in Neuroinformatics, 7, 14. https://doi.org/10.3389/fninf.2013.00014
59.
ZhouH.LiS.HuangJ.YangJ.WangA.ZhangM. (2023). Sound-induced flash illusions at different spatial locations were affected by personality traits. Attention, Perception, & Psychophysics, 85(2), 463–473. https://doi.org/10.3758/s13414-022-02638-5
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
