Do novel linguistic labels have privileged access to attentional resources compared to non-linguistic labels? This study explores this possibility through two experiments with a training and an attentional overlap task. Experiment 1 investigates how novel label and object-only stimuli influence resource allocation and disengagement of visual attention. Experiment 2 tests the impact of linguistic information on visual attention by comparing novel tones and labels. Because disengagement of attention is affected both by the salience of the perceptual stimulus and by the degree of familiarity with the stimulus to be disengaged from, we compared pupil size variations and saccade latency under different test conditions: (a) consistent with (i.e., identical to) the training; (b) inconsistent with the training (i.e., with an altered feature), and (c) deprived of one feature only in Experiment 1. Experiment 1 indicated a general consistency advantage (and deprived disadvantage) driven by linguistic label-object pairs compared to object-only stimuli. Experiment 2 revealed that tone-object pairs led to higher pupil dilation and longer saccade latency than linguistic label-object pairs. Our results suggest that novel linguistic labels preferentially impact the early orienting of attention.
AlsiusA.NavarraJ.CampbellR.Soto-FaracoS. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15(9), 839–843. https://doi.org/10.1016/j.cub.2005.03.046
2.
BaayenH.VasishthS.KlieglR.BatesD. (2017). The cave of shadows: Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234. https://doi.org/10.1016/j.jml.2016.11.006
3.
BaayenR. H.DavidsonD. J.BatesD. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/j.jml.2007.12.005
4.
BalabanM. T.WaxmanS. R. (1997). Do linguistic labels facilitate object categorization in 9-month-old infants?Journal of Experimental Child Psychology, 64(1), 3–26. https://doi.org/10.1006/jecp.1996.2332
5.
BaldwinD. A. (1991). Infants’ contribution to the achievement of joint reference. Child Development, 62(5), 875–890.
6.
BatesD.MaechlerM.BolkerB.WalkerS.ChristensenR. H. B.SingmannH.KrivitskyP. N. (2015). Package “lme4.”Convergence, 12(1), Article 2.
BergelsonE.SwingleyD. (2012). At 6–9 months, human infants know the meanings of many common nouns. Proceedings of the National Academy of Sciences, 109(9), 3253–3258. https://doi.org/10.1073/pnas.1113380109
9.
BlalockL. D. (2015). Stimulus familiarity improves consolidation of visual working memory representations. Attention, Perception, & Psychophysics, 77(4), 1143–1158. https://doi.org/10.3758/s13414-014-0823-z
BoutonnetB.LupyanG. (2015). Linguistic labels jump-start vision: A label advantage in object recognition. Journal of Neuroscience, 35(25), 9329–9335. https://doi.org/10.1523/JNEUROSCI.5111-14.2015
12.
BollenK. A. (1989). A new incremental fit index for general struc-tural equation models. Sociological Methods & Research, 17(3), 303–316. https://doi.org/10.1177/0049124189017003004
13.
BrissonJ.MainvilleM.MaillouxD.BeaulieuC.SerresJ.SiroisS. (2013). Pupil diameter measurement errors as a function of gaze direction in corneal reflection eyetrackers. Behavior Research Methods, 45(4), 1322–1331. https://doi.org/10.3758/s13428-013-0327-0
14.
CareyS.BartlettE. (1978). Acquiring a single new word. Child Language Development, 15, 17–29.
15.
CorbettaM.KincadeJ. M.OllingerJ. M.McAvoyM. P.ShulmanG. L. (2000). Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nature Neuroscience, 3(3), 292–297.
16.
DavisC. P.MorrowH. M.LupyanG. (2019). What does a horgous look like? Nonsense words elicit meaningful drawings. Cognitive Science, 43(10), Article e12791.
17.
DebenerS.KrancziochC.HerrmannC. S.EngelA. K. (2002). Auditory novelty oddball allows reliable distinction of top–down and bottom–up processes of attention. International Journal of Psychophysiology, 46(1), 77–84. https://doi.org/10.1016/S0167-8760(02)00072-7
18.
DevaneyK. J.RosenM. L.LevinE. J.SomersD. C. (2019). Identification of visual attentional regions of the temporoparietal junction in individual subjects using a Vivid, Novel Oddball Paradigm. Frontiers in Human Neuroscience, 13, Article 424.
19.
DiQuattroN. E.SawakiR.GengJ. J. (2014). Effective connectivity during feature-based attentional capture: Evidence against the attentional reorienting hypothesis of TPJ. Cerebral Cortex, 24(12), 3131–3141.
20.
DriverJ.EimerM.MacalusoE.Van VelzenJ. L. (2004). The neurobiology of human spatial attention. In KanwisherN.DuncanJ. (Eds.), Functional neuroimaging of visual cognition. Attention & performance XX (pp. 267–300). Oxford University Press. https://doi.org/10.1162/jocn_a_01283
FairhallS. L.MacalusoE. (2009). Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. European Journal of Neuroscience, 29(6), 1247–1257. https://doi.org/10.1111/j.1460-9568.2009.06688.x
24.
FensonL.DaleP. S.ReznickJ. S.BatesE.ThalD. J.PethickS. J.. . . StilesJ. (1994). Variability in early communicative development. Monographs of the Society for Research in Child Development, 85, i+iii-v+i–185. https://doi.org/10.2307/1166093
25.
ForsytheA.MulhernG.SaweyM. (2008). Confounds in pictorial sets: The role of complexity and familiarity in basic-level picture processing. Behavior Research Methods, 40(1), 116–129. https://doi.org/10.3758/BRM.40.1.116
26.
FrensM. A.Van OpstalA. J.Van der WilligenR. F. (1995). Spatial and temporal factors determine auditory-visual interactions in human saccade eye movements. Perception & Psychophysics, 57(6), 802–816. https://doi.org/10.3758/BF03206796
27.
GaneaP. A.ShuttsK.SpelkeE. S.DeLoacheJ. S. (2007). Thinking of things unseen: Infants’ use of language to update mental representations. Psychological Science, 18(8), 734–739. https://doi.org/10.1111/j.1467-9280.2007.01968.x
GengJ. J.VosselS. (2013). Re-evaluating the role of TPJ in attentional control: Contextual updating?Neuroscience & Biobehavioral Reviews, 37(10), 2608–2620.
30.
GriffinZ. M.OppenheimerD. M. (2006). Speakers gaze at objects while preparing intentionally inaccurate labels for them. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(4), 943–948. https://doi.org/10.1037/0278-7393.32.4.943
HoC.SpenceC. (2006). Verbal interface design: Do verbal directional cues automatically orient visual spatial attention?Computers in Human Behavior, 22(4), 733–748. https://doi.org/10.1016/j.chb.2005.12.008
36.
HoodB. M.AtkinsonJ. (1993). Disengaging visual attention in the infant and adult. Infant Behavior and Development, 16(4), 405–422. https://doi.org/10.1016/0163-6383(93)80001
37.
HorstJ. S.HoutM. C. (2016). The Novel Object and Unusual Name (NOUN) Database: A collection of novel images for use in experimental research. Behavior Research Methods, 48(4), 1393–1409. https://doi.org/10.3758/BRM.42.1.351
38.
IvanovaA. A.HoferM. (2020, May28). Linguistic overhypotheses in category learning: Explaining the label advantage effect. https://doi.org/10.31234/osf.io/x9e4z
39.
JaintaS.VernetM.YangQ.KapoulaZ. (2011). The pupil reflects motor preparation for saccades-even before the eye starts to move. Frontiers in Human Neuroscience, 5, Article 97. https://doi.org/10.3389/fnhum.2011.00097
40.
Karmiloff-SmithA. (2015). An alternative to domain-general or domain-specific frameworks for theorizing about human evolution and ontogenesis. AIMS Neuroscience, 2(2), Article 91.
41.
KenwardB.KochF. S.ForssmanL.BrehmJ.TidemannI.SundqvistA.. . . GredebäckG. (2017). Saccadic reaction times in infants and adults: Spatiotemporal factors, gender, and interlaboratory variation. Developmental Psychology, 53(9), 1750–1764. https://doi.org/10.1037/dev0000338
42.
KnofczynskiG. T.MundfromD. (2008). Sample sizes when using multiple linear regression for prediction. Educational and Psychological Measurement, 68(3), 431–442. https://doi.org/10.1177/0013164407310131
43.
KonkleT.BradyT. F.AlvarezG. A.OlivaA. (2010). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General, 139(3), 558–578. https://doi.org/10.1037/a0019165
44.
KramerS. E.LorensA.ConinxF.ZekveldA. A.PiotrowskaA.SkarzynskiH. (2012). Processing load during listening: The influence of task characteristics on the pupil response. Language and Cognitive Processes, 28(4), 426–442. https://doi.org/10.1080/01690965.2011.642267
45.
KrebsR. M.ParkH. R.BombekeK.BoehlerC. N. (2018). Modulation of locus coeruleus activity by novel oddball stimuli. Brain Imaging and Behavior, 12(2), 577–584.
46.
KucewiczM. T.DolezalJ.KremenV.BerryB. M.MillerL. R.MageeA. L.. . . WorrellG. A. (2018). Pupil size reflects successful encoding and recall of memory in humans. Scientific Reports, 8(1), Article 4949. https://doi.org/s41598-018-23197-6
47.
LaengB.SiroisS.GredebäckG. (2012). Pupillometry: A window to the preconscious?Perspectives on Psychological Science, 7(1), 18–27. https://doi.org/10.1177/1745691611427305
48.
LepsienJ.NobreA. C. (2006). Cognitive control of attention in the human brain: Insights from orienting attention to mental representations. Brain Research, 1105(1), 20–31. https://doi.org/10.1016/j.brainres.2006.03.033
49.
LiC.HuZ.YangJ. (2020). Rapid acquisition through fast mapping: Stable memory over time and role of prior knowledge. Learning & Memory, 27(5), 177–189. https://doi.org/10.1101/lm.050138.119
50.
LiebenthalE.BinderJ. R.PiorkowskiR. L.RemezR. E. (2003). Short-term reorganization of auditory analysis induced by phonetic experience. Journal of Cognitive Neuroscience, 15(4), 549–558. https://doi.org/10.1162/089892903321662930
LottoL.SurianL.JobR. (2010). Objective age of acquisition for 223 Italian linguistic labels: Norms and effects on picture naming speed. Behavior Research Methods, 42(1), 126–133. https://doi.org/10.3758/BRM.42.1.126
LupyanG.Thompson-SchillS. L. (2012). The evocative power of linguistic labels: Activation of concepts by verbal and nonverbal means. Journal of Experimental Psychology: General, 141(1), 170–186. https://doi.org/10.1037/a0024904
55.
LupyanG.WardE. J. (2013). Language can boost otherwise unseen objects into visual awareness. Proceedings of the National Academy of Sciences, 110(35), 14196–14201. https://doi.org/10.1073/pnas.1303312110
56.
MathôtS. (2018). Pupillometry: Psychology, Physiology, and Function. Journal of Cognition, 1(1), Article 16. http://doi.org/10.5334/joc.18
57.
MathôtS.SchreijD.TheeuwesJ. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44(2), 314–324. https://doi.org/10.3758/s13428-011-0168-7
58.
MathôtS.Van der LindenL.GraingerJ.VituF. (2013). The pupillary light response reveals the focus of covert visual attention. PLOS ONE, 8(10), Article e78168. https://doi.org/10.1371/journal.pone.0078168
59.
MulattiC.TreccaniB.JobR. (2014). The role of the sound of objects in object identification: Evidence from picture naming. Frontiers in Psychology, 5, Article 1139. https://doi.org/10.3389/fpsyg.2014.01139
60.
OsinaM. A.SaylorM. M.GaneaP. A. (2018). Infants use category label knowledge to interpret absent reference. Infancy, 23(5), 650–673.
61.
OstarekM.HuettigF. (2017). Spoken words can make the invisible visible—Testing the involvement of low-level visual representations in spoken word processing. Journal of Experimental Psychology: Human Perception and Performance, 43(3), 499–508. https://doi.org/10.1037/xhp0000313
RuthruffE.RemingtonR. W.JohnstonJ. C. (2001). Switching between simple cognitive tasks: The interaction of top-down and bottom-up factors. Journal of Experimental Psychology: Human Perception and Performance, 27(6), 1404–1419. https://doi.org/10.1037/0096-1523.27.6.1404
68.
SadaghianiS.MaierJ. X.NoppeneyU. (2009). Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. Journal of Neuroscience, 29(20), 6490–6499. https://doi.org/10.1523/JNEUROSCI.5437-08.2009
69.
SaylorM. M. (2004). Twelve-and 16-month-old infants recognize properties of mentioned absent things. Developmental Science, 7(5), 599–611.
70.
SchacterD. L.DobbinsI. G.SchnyerD. M. (2004). Specificity of priming: A cognitive neuroscience perspective. Nature Reviews Neuroscience, 5(11), 853–862. https://doi.org/10.1038/nrn1534
71.
SchindelR.RowlandsJ.ArnoldD. H. (2011). The oddball effect: Perceived duration and predictive coding. Journal of Vision, 11(2), Article 17. https://doi.org/10.1167/11.2.17
72.
SinnettS.Soto-FaracoS.SpenceC. (2008). The co-occurrence of multisensory competition and facilitation. Acta Psychologica, 128(1), 153–161. https://doi.org/10.1016/j.actpsy.2007.12.002
SpiegelC.HalberdaJ. (2011). Rapid fast-mapping abilities in 2-year-olds. Journal of Experimental Child Psychology, 109(1), 132–140.
75.
SpinelliG.SulpizioS.BuraniC. (2017). Q2Stress: A database for multiple cues to stress assignment in Italian. Behavior research methods, 49(6), 2113–2126.
76.
SuzukiY.MinamiT.NakauchiS. (2018). Association between pupil dilation and implicit processing prior to object recognition via insight. Scientific Reports, 8(1), Article 6874. https://doi.org/10.1038/s41598-018-25207-z
TalsmaD.DotyT. J.WoldorffM. G. (2007). Selective attention and audiovisual integration: Is attending to both modalities a prerequisite for early integration?Cerebral Cortex, 17(3), 679–690. https://doi.org/0.1093/cercor/bhk016
79.
TseP. U.IntriligatorJ.RivestJ.CavanaghP. (2004). Attention and the subjective expansion of time. Perception & Psychophysics, 66, 1171–1189. https://doi.org/10.3758/BF03196844
80.
van RijJ.HendriksP.van RijnH.BaayenR. H.WoodS. N. (2019). Analyzing the time course of pupillometric data. Trends in Hearing, 23, 2331216519832483. https://doi.org/10.1177/2331216519832483
81.
VossJ. L.SchendanH. E.PallerK. A. (2010). Finding meaning in novel geometric shapes influences electrophysiological correlates of repetition and dissociates perceptual and conceptual priming. NeuroImage, 49(3), 2879–2889.
82.
WagenmakersE.-J.FarrellS. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11(1), 192–196. https://doi.org/10.3758/BF03206482
83.
WardE. J.LupyanG. (2011). Linguistic penetration of suppressed visual representations. Journal of Vision, 11, Article 322. https://doi.org/10.1167/11.11.322
84.
WaxmanS. R.GelmanS. A. (2009). Early linguistic label-learning entails reference, not merely associations. Trends in Cognitive Sciences, 13(6), 258–263.
85.
WillitsJ. A.WojcikE. H.SeidenbergM. S.SaffranJ. R. (2013). Toddlers activate lexical semantic knowledge in the absence of visual referents: Evidence from auditory priming. Infancy, 18(6), 1053–1075.
86.
WolfE. J.HarringtonK. M.ClarkS. L.MillerM. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934. https://doi.org/10.1177/0013164413495237
87.
WolfeJ. M.HorowitzT. S. (2004). What attributes guide the deployment of visual attention and how do they do it?Nature Reviews Neuroscience, 5(6), 495–501. https://doi.org/10.1038/nrn1411
88.
WoodS. N. (2011). Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. In Journal of the Royal Statistical Society (B), 73(1), 3–36.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
