Although humans are almost constantly exposed to stimuli from multiple sensory modalities during daily life, the processes by which we learn to integrate information from multiple senses to acquire knowledge of multisensory objects are not well understood. Here, we present results of a novel audio – visual statistical learning procedure where participants are passively exposed to a rapid serial presentation of arbitrary audio — visual pairings (comprised of artificial/synthetic audio and visual stimuli). Following this exposure, participants were tested with a two-interval forced-choice procedure in which their degree of familiarity with the experienced audio-visual pairings was evaluated against novel audio — visual combinations drawn from the same stimulus set. Our results show that subjects acquire knowledge of visual — visual, audio — audio, and audio — visual stimulus associations and that the learning of these types of associations occurs in an independent manner.
Get full access to this article
View all access options for this article.
References
1.
BeerA LRoderB, 2005“Attending to visual or auditory motion affects perception within and across modalities: an event-related potential study”European Journal of Neuroscience211116–1130
2.
BlasdelG GMitchellD EMuirD WPettigrewJ D, 1977“A physiological and behavioural study in cats of the effect of early visual experience with contours of a single orientation”Journal of Physiology265615–636
BusseLRobertsK CCristR EWeissmanD HWoldorffM G, 2005“The spread of attention across modalities and space in a multisensory object”Proceedings of the National Academy of Sciences of the USA10218751–18756
5.
CalvertGSpenceCSteinB E, 2004The Handbook of Multisensory Processes (Cambridge, MA: MIT Press)
6.
ChangE FBaoSImaizumiKSchreinerC EMerzenichM M, 2005“Development of spectral and temporal response selectivity in the auditory cortex”Proceedings of the National Academy of Sciences of the USA10216460–16465
7.
ChunM M, 2000“Contextual cueing of visual attention”Trends in Cognitive Sciences4170–178
8.
ConwayC MChristiansenM H, 2005“Modality-constrained statistical learning of tactile, visual, and auditory sequences”Journal of Experimental Psychology: Learning, Memory, and Cognition3124–39
9.
ConwayC MChristiansenM H, 2006“Statistical learning within and between modalities: pitting abstract against stimulus-specific representations”Psychological Science17905–912
10.
DasAFrancaJ GGattassRKaasJ HNicolelisM ATimo-IariaCVargasC DWeinbergerN MVolchanE, 2001“The brain decade in debate: VI. Sensory and motor maps: dynamics and plasticity”Brazilian Journal of Medical and Biological Research341497–1508
11.
DinseH RRagertPPlegerBSchwenkreisPTegenthoffM, 2003“Pharmacological modulation of perceptual learning and associated cortical reorganization”Science30191–94
12.
DriverJ, 1996“Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading”Nature38166–68
13.
DriverJSpenceC, 1998“Crossmodal attention”Current Opinion in Neurobiology8245–253
14.
DuncanJMartensSWardR, 1997“Restricted attentional capacity within but not between sensory modalities”Nature387808–810
15.
FiserJAslinR N, 2001“Unsupervised statistical learning of higher-order spatial structures from visual scenes”Psychological Science12499–504
16.
FiserJAslinR N, 2002“Statistical learning of higher-order temporal structure from visual shape sequences”Journal of Experimental Psychology: Learning, Memory, and Cognition28458–467
17.
FrenkelM YSawtellN BDiogoA CYoonBNeveR LBearM F, 2006“Instructive effect of visual experience in mouse visual cortex”Neuron51339–349
18.
GhazanfarA ASchroederC E, 2006“Is neocortex essentially multisensory?”Trends in Cognitive Sciences10278–285
19.
GoldstoneR L, 1998“Perceptual learning”Annual Review of Psychology49585–612
20.
GouletSMurrayE A, 2001“Neural substrates of crossmodal association memory in monkeys: the amygdala versus the anterior rhinal cortex”Behavioral Neuroscience115271–284
21.
GrossbergS, 1976“On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems”Biological Cybernetics21145–159
22.
GrossbergSSeitzA, 2003“Laminar development of receptive fields, maps and columns in visual cortex: the coordinating role of the subplate”Cerebral Cortex13852–863
23.
HowardI PTempletonW B, 1966Human Spatial Orientation (London: Wiley and Sons)
24.
HubelD HWieselT N, 1964“Effects of monocular deprivation in kittens”Naunyn-Schmeidebergs Archiv für Experimentelle Pathologie und Pharmacologie248492–497
25.
KingA JMooreD R, 1991“Plasticity of auditory maps in the brain”Trends in Neuroscience1431–37
26.
KingA JParsonsC HMooreD R, 2000“Plasticity in the neural coding of auditory space in the mammalian brain”Proceedings of the National Academy of Sciences of the USA9711821–11828
27.
KohonenT, 1982“Self-organized formation of topologically correct feature maps”Biological Cybernetics4359–69
28.
LaceySCampbellC, 2006“Mental representation in visual/haptic crossmodal memory: evidence from interference effects”Quarterly Journal of Experimental Psychology59361–376
29.
LaudauerT K, 1986“How much do people remember? Some estimates of the quantity of learned information in long-term memory”Cognitive Science10477–493
30.
McGurkHMacDonaldJ W, 1976“Hearing lips and seeing voices”Nature264746–748
31.
von der MalsburgC, 1973“Self-organization of orientation sensitive cells in the striate cortex”Kybernetik1485–100
32.
PelliD G, 1997“The VideoToolbox software for visual psychophysics: transforming numbers into movies”Spatial Vision10437–442
33.
SaffranJ RJohnsonE KAslinR NNewportE L, 1999“Statistical learning of tone sequences by human infants and adults”Cognition7027–52
34.
SeitzA RDinseH R, 2007“A common framework for perceptual learning”Current Opinion in Neurobiology17148–153
35.
SeitzAWatanabeT, 2005“A unified model for perceptual learning”Trends in Cognitive Sciences9329–334
36.
SeitzA RYamagishiNWernerBGodaNKawatoMWatanabeT, 2005“Task-specific disruption of perceptual learning”Proceedings of the National Academy of Sciences of the USA10214895–14900
ShamsLKamitaniYShimojoS, 2000“What you see is what you hear”Nature408788
39.
ShimojoSShamsL, 2001“Sensory modalities are not separate modalities: plasticity and interactions”Current Opinion in Neurobiology11505–509
40.
SingerWFreemanBRauscheckerJ, 1981“Restriction of visual experience to a single orientation affects the organization of orientation columns in cat visual cortex. A study with deoxyglucose”Experimental Brain Research41199–215
41.
Soto-FaracoSLyonsJGazzanigaMSpenceCKingstoneA, 2002“The ventriloquist in motion: Illusory capture of dynamic information across sensory modalities”Cognitive Brain Research14139–146
42.
SpenceCDriverJ, 1996“Audiovisual links in endogenous covert spatial attention”Journal of Experimental Psychology: Human Perception and Performance221005–1030
43.
SpenceCPavaniFMaravitaAHolmesN, 2004“Multisensory contributions to the 3-D representation of visuotactile peripersonal space in humans: evidence from the crossmodal congruency task”Journal of Physiology (Paris)98171–189
44.
TanabeH CHondaMSadatoN, 2005“Functionally segregated neural substrates for arbitrary audiovisual paired-association learning”Journal of Neuroscience256409–6418
