Even if the scene before our eyes remains static for some time, we might explore it differently compared with how we examine static images, which are commonly used in studies on visual attention. Here we show experimentally that the top-down expectation of changes in natural scenes causes clearly distinguishable gaze behavior for visually identical scenes. We present free-viewing eye-tracking data of 20 healthy adults on a new video dataset of natural scenes, each mapped for its potential for change (PfC) in independent ratings. Observers looking at frozen videos looked significantly more often at the parts of the scene with a high PfC compared with static images, with substantially higher interobserver coherence. This viewing difference peaked right before a potential movement onset. Established concepts like object animacy or salience alone could not explain this finding. Images thus conceal experience-based expectations that affect gaze behavior in the potentially dynamic real world.
AçıkA.BartelA.KönigP. (2014). Real and implied motion at the center of gaze. Journal of Vision, 14(1), Article 2. https://doi.org/10.1167/14.1.2
2.
AndersonN. C.OrtE.KruijneW.MeeterM.DonkM. (2015). It depends on when you look at it: Salience influences eye movements in natural scene viewing and search early in time. Journal of Vision, 15(5), Article 9. https://doi.org/10.1167/15.5.9
3.
BakerD. H.VilidaiteG.LygoF. A.SmithA. K.FlackT. R.GouwsA. D.AndrewsT. J. (2021). Power contours: Optimising sample size and precision in experimental psychology and human neuroscience. Psychological Methods, 26(3), 295–314.
4.
BinaeeK.DiazG. (2019). Movements of the eyes and hands are coordinated by a common predictive strategy. Journal of Vision, 19(12), Article 3. https://doi.org/10.1167/19.12.3
5.
BrodaM. D.HaddadT.de HaasB. (2023). Quick, eyes! Isolated upper face regions but not artificial features elicit rapid saccades. Journal of Vision, 23(2), Article 5. https://doi.org/10.1167/jov.23.2.5
6.
CalvilloD. P.HawkinsW. C. (2016). Animate objects are detected more frequently than inanimate objects in inattentional blindness tasks independently of threat. The Journal of General Psychology, 143(2), 101–115.
7.
CalvilloD. P.JacksonR. E. (2014). Animacy, perceptual load, and inattentional blindness. Psychonomic Bulletin & Review, 21, 670–675.
8.
CarmiR.IttiL. (2006). Visual causes versus correlates of attentional selection in dynamic scenes. Vision Research, 46(26), 4333–4345.
9.
Carneiro MoritaV.SoutoD.MassonG. S.MontagniniA. (2023). Anticipatory smooth eye movements scale with the probability of visual motion: Role of target speed and acceleration [Preprint]. bioRxiv.
10.
CarrascoM. (2011). Visual attention: The past 25 years. Vision Research, 51(13), 1484–1525.
11.
CornelissenF. W.PetersE. M.PalmerJ. (2002). The Eyelink Toolbox: Eye tracking with MATLAB and the Psychophysics Toolbox. Behavior Research Methods, Instruments, & Computers, 34(4), 613–617.
12.
DamasseJ.-B.PerrinetL. U.MadelainL.MontagniniA. (2018). Reinforcement effects in anticipatory smooth eye movements. Journal of Vision, 18(11), Article 14. https://doi.org/10.1167/18.11.14
13.
DarA. H.WagnerA. S.HankeM. (2021). Remodnav: Robust eye-movement classification for dynamic stimulation. Behavior Research Methods, 53(1), 399–414.
14.
de HaasB.IakovidisA. L.SchwarzkopfD. S.GegenfurtnerK. R. (2019). Individual differences in visual salience vary along semantic dimensions. Proceedings of the National Academy of Sciences, USA, 116(24), 11687–11692.
15.
DiazG.CooperJ.RothkopfC.HayhoeM. (2013). Saccades to future ball location reveal memory-based prediction in a virtual-reality interception task. Journal of Vision, 13(1), Article 20. https://doi.org/10.1167/13.1.20
16.
DiazG. J.FajenB. R.PhillipsF. (2012). Anticipation from biological motion: The goalkeeper problem. Journal of Experimental Psychology: Human Perception and Performance, 38(4), 848–864.
17.
DonkM.Van ZoestW. (2008). Effects of salience are short-lived. Psychological Science, 19(7), 733–739.
18.
DorrM.BexP. J. (2013). Peri-saccadic natural vision. Journal of Neuroscience, 33(3), 1211–1217.
19.
DorrM.MartinetzT.GegenfurtnerK. R.BarthE. (2010). Variability of eye movements when viewing dynamic natural scenes. Journal of Vision, 10(10), Article 28. https://doi.org/10.1167/10.10.28
FookenJ.KreyenmeierP.SperingM. (2021). The role of eye movements in manual interception: A mini-review. Vision Research, 183, 81–90.
22.
GoettkerA.BorgerdingN.LeeskeL.GegenfurtnerK. R. (2023). Cues for predictive eye movements in naturalistic scenes. Journal of Vision, 23(10), Article 12. https://doi.org/10.1167/jov.23.10.12
23.
GoettkerA.PidaparthyH.BraunD. I.ElderJ. H.GegenfurtnerK. R. (2021). Ice hockey spectators use contextual cues to guide predictive eye movements. Current Biology, 31(16), R991–R992.
24.
HeK.GkioxariG.DollárP.GirshickR. (2017, October 22-29). Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2961–2969). IEEE. doi: 10.1109/ICCV.2017.322
25.
HendersonJ. M. (2003). Human gaze control during real-world scene perception. Trends in Cognitive Sciences, 7(11), 498–504.
26.
HendersonJ. M.HayesT. R. (2017). Meaning-based guidance of attention in scenes as revealed by meaning maps. Nature Human Behaviour, 1(10), 743–747.
27.
HershlerO.HochsteinS. (2005). At first sight: A high-level pop out effect for faces. Vision Research, 45(13), 1707–1724.
28.
IttiL. (2005). Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition, 12(6), 1093–1123.
29.
IttiL.KochC.NieburE. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.
30.
KaakinenJ. K. (2021). What can eye movements tell us about visual perception processes in classroom contexts? Commentary on a special issue. Educational Psychology Review, 33(1), 169–179.
31.
KirchnerH.ThorpeS. J. (2006). Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited. Vision Research, 46(11), 1762–1776.
32.
KirkorianH. L.AndersonD. R. (2017). Anticipatory eye movements while watching continuous action across shots in video sequences: A developmental study. Child Development, 88(4), 1284–1301.
33.
KleinerM.BrainardD.PelliD. (2007). What’s new in psychtoolbox-3?Perception, 36(14), 1–16.
KümmererM.BethgeM.WallisT. S. (2022). DeepGaze III: Modeling free-viewing human scanpaths with deep learning. Journal of Vision, 22(5), Article 7. https://doi.org/10.1167/jov.22.5.7
36.
LandM. F.FurneauxS. (1997). The knowledge base of the oculomotor system. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 352(1358), 1231–1239.
37.
LandM. F.HayhoeM. (2001). In what ways do eye movements contribute to everyday activities?Vision Research, 41(25–26), 3559–3565.
38.
LinardosA.KümmererM.PressO.BethgeM. (2021, October 10-17). DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 12919–12928). IEEE. doi: 10.1109/ICCV48922.2021.01268
39.
LinkaM.de HaasB. (2023). Detection, inspection, return: A functional classification of fixations in complex scenes [Preprint]. PsyArXiv.
40.
LoffingF.Cañal-BrulandR. (2017). Anticipation in sport. Current Opinion in Psychology, 16, 6–11.
41.
MitalP. K.SmithT. J.HillR. L.HendersonJ. M. (2011). Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation, 3, 5–24.
42.
NewJ.CosmidesL.ToobyJ. (2007). Category-specific attention for animals reflects ancestral priorities, not expertise. Proceedings of the National Academy of Sciences, USA, 104(42), 16598–16603.
NyströmM.HolmqvistK. (2010). An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behavior Research Methods, 42(1), 188–204.
45.
PedziwiatrM. A.KümmererM.WallisT. S.BethgeM.TeufelC. (2021). Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations. Cognition, 206, Article 104465.
46.
PosnerM. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32(1), 3–25.
RothN.RolfsM.HellwichO.ObermayerK. (2023). Objects guide human gaze behavior in dynamic real-world scenes. PLOS Computational Biology, 19(10), 1–39. https://doi.org/10.1371/journal.pcbi.1011512
49.
RothkegelL. O.TrukenbrodH. A.SchüttH. H.WichmannF. A.EngbertR. (2017). Temporal evolution of the central fixation bias in scene viewing. Journal of Vision, 17(13), Article 3. https://doi.org/10.1167/17.13.3
50.
SmithT. J.MitalP. K. (2013). Attentional synchrony and the influence of viewing task on gaze behavior in static and dynamic scenes. Journal of Vision, 13(8), Article 16. https://doi.org/10.1167/13.8.16
51.
SperingM.CarrascoM. (2015). Acting without seeing: Eye movements reveal visual processing without awareness. Trends in Neurosciences, 38(4), 247–258.
52.
SperingM.SchützA. C.BraunD. I.GegenfurtnerK. R. (2011). Keep your eyes on the ball: Smooth pursuit eye movements enhance prediction of visual motion. Journal of Neurophysiology, 105(4), 1756–1767.
53.
StewartE. E.FlemingR. W. (2023). The eyes anticipate where objects will move based on their shape. Current Biology, 33(17), R894–R895.
54.
SummerfieldC.De LangeF. P. (2014). Expectation in perceptual decision making: Neural and computational mechanisms. Nature Reviews Neuroscience, 15(11), 745–756.
55.
VigE.DorrM.MartinetzT.BarthE. (2011). Eye movements show optimal average anticipation with natural dynamic scenes. Cognitive Computation, 3, 79–88.
56.
VoM. L.-H.BoettcherS. E.DraschkowD. (2019). Reading scenes: How scene grammar guides attention and aids perception in real-world environments. Current Opinion in Psychology, 29, 205–210.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
