This study used the looking-at-nothing phenomenon to explore situation awareness (SA) and the effects of working memory (WM) load in driving situations.
Background
While driving, people develop a mental representation of the environment. Since errors in retrieving information from this representation can have fatal consequences, it is essential for road safety to investigate this process. During retrieval, people tend to fixate spatial positions of visually encoded information, even if it is no longer available at that location. Previous research has shown that this “looking-at-nothing” behavior can be used to trace retrieval processes.
Method
In a video-based laboratory experiment with 2 (WM) x 3 (SA level) within-subjects design, participants (N = 33) viewed a reduced screen and evaluated auditory statements relating to different SA levels on previously seen dynamic traffic scenarios while eye movements were recorded.
Results
When retrieving information, subjects more frequently fixated emptied spatial locations associated with the information relevant for the probed SA level. The retrieval of anticipations (SA level 3) in contrast to the other SA level information resulted in more frequent gaze transitions that corresponded to the spatial dynamics of future driving behavior.
Conclusion
The results support the idea that people build a visual-spatial mental image of a driving situation. Different gaze patterns when retrieving level-specific information indicate divergent retrieval processes.
Application
Potential applications include developing new methodologies to assess the mental representation and SA of drivers objectively.
AltmannG. T. M. (2004). Language-mediated eye movements in the absence of a visual world: The “blank screen paradigm”. Cognition, 93(2), B79–B87. https://doi.org/10.1016/j.cognition.2004.02.005
2.
BaayenR. H.DavidsonD. J.BatesD. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. https://doi.org/10.1016/j.jml.2007.12.005
3.
BalkS. A.MooreK. S.SteeleJ. E.SpearmanW. J.DuchowskiA. T. (2010). Mobile phone use in a driving simulation task: Differences in eye movements. Journal of Vision, 6(6), 872. https://doi.org/10.1167/6.6.872
4.
BarrD. J.LevyR.ScheepersC.TilyH. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. https://doi.org/10.1016/j.jml.2012.11.001
5.
BaumannM.KremsJ. F. (2007). Situation Awareness and Driving: A Cognitive Model. In CacciabueP. C. (Ed.), Modelling Driver Behaviour in Automotive Enviroments. Critical Issues in Driver Interactions with Intelligent Transport Systems. London: Springer.
6.
BaumannM.KremsJ. K. (2009). A comprehension based cognitive model of situation awareness. In DuffyV. D. (Ed.), Digital human modeling (Vol. 5620, pp. 192–201). Springer. https://doi.org/10.1007/978-3-642-02809-0_21
7.
ChiquetS.MartarelliC. S.MastF. W. (2020). Eye movements to absent objects during mental imagery and visual memory in immersive virtual reality. Virtual Reality, 25(3), 655–667. https://doi.org/10.1007/s10055-020-00478-y
8.
DursoF. T.RawsonK. A.GirottoS. (2007). Comprehension and situation awareness. In DursoF. T.NickersonR. S.DumaisS. T., et al. (Eds.), Handbook of applied cognition (2nd ed., pp. 163–193). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470713181.ch7
9.
EndsleyM. R. (1988). Situation awareness global assessment technique (SAGAT). In Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, Dayton, OH, USA, 23–27 May 1988. 789–795. https://doi.org/10.1109/NAECON.1988.195097
10.
EndsleyM. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32–64. https://doi.org/10.1518/001872095779049543
11.
EndsleyM. R. (2020). The divergence of objective and subjective situation awareness: A meta-analysis. Journal of Cognitive Engineering and Decision Making, 14(1), 34–53. https://doi.org/10.1177/1555343419874248
12.
FerreiraF.ApelJ.HendersonJ. M. (2008). Taking a new look at looking at nothing. Trends in Cognitive Sciences, 12(11), 405–410. https://doi.org/10.1016/j.tics.2008.07.007
13.
GugertyL. J. (1997). Situation awareness during driving: Explicit and implicit knowledge in dynamic spatial memory. Journal of Experimental Psychology: Applied, 3(1), 42–66. https://doi.org/10.1037/1076-898X.3.1.42
14.
HooverM. A.RichardsonD. C. (2008). When facts go down the rabbit hole: Contrasting features and objecthood as indexes to memory. Cognition, 108(2), 533–542. https://doi.org/10.1016/j.cognition.2008.02.011
15.
HumphreyK.UnderwoodG. (2008). Fixation sequences in imagery and in recognition during the processing of pictures of real-world scenes. Journal of Eye Movement Research, 2(2), 1–15. https://doi.org/10.16910/jemr.2.2.3
16.
JohanssonR.HolsanovaJ.HolmqvistK. (2006). Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness. Cognitive Science, 30(6), 1053–1079. https://doi.org/10.1207/s15516709cog0000_86
17.
JohanssonR.HolsanovaJ.HolmqvistK. (2010). Eye movements during mental imagery are not reenactments of perception. In OhlssonS.CatramboneR. (Eds.), Cognition in flux (pp. 1968–1973). Cognitive Science Society, Inc.
18.
JohanssonR.JohanssonM. (2014). Look here, eye movements play a functional role in memory retrieval. Psychological Science, 25(1), 236–242. https://doi.org/10.1177/0956797613498260
19.
KintschW. (1998). Comprehension: A paradigm for cognition. Cambridge University Press.
20.
KlichowiczA.StrehlauS.BaumannM. R.KremsJ. F.RosnerA. (2020). Tracing current explanations in memory: A process analysis based on eye-tracking. Quarterly Journal of Experimental Psychology (2006), 73(10), 1703–1717. https://doi.org/10.1177/1747021820922509
21.
KumcuA.ThompsonR. L. (2020). Less imageable words lead to more looks to blank locations during memory retrieval. Psychological Research, 84(3), 667–684. https://doi.org/10.1007/s00426-018-1084-6
22.
LaengB.BloemI. M.D’AscenzoS.TommasiL. (2014). Scrutinizing visual images: The role of gaze in mental imagery and memory. Cognition, 131(2), 263–283. https://doi.org/10.1016/j.cognition.2014.01.003
23.
MaR.KaberD. B. (2005). Situation awareness and workload in driving while using adaptive cruise control and a cell phone. International Journal of Industrial Ergonomics, 35(10), 939–953. https://doi.org/10.1016/j.ergon.2005.04.002
24.
MartarelliC. S.MastF. W. (2011). Preschool children’s eye-movements during pictorial recall. The British Journal of Developmental Psychology, 29(3), 425–436. https://doi.org/10.1348/026151010X495844
25.
MühlK.StollT.BaumannM. (2020). Look ahead: Understanding cognitive anticipatory processes based on situational characteristics in dynamic traffic situations. IET Intelligent Transport Systems, 14(4), 233–240. https://doi.org/10.1049/iet-its.2018.5557
26.
MuhrerE.VollrathM. (2011). The effect of visual and cognitive distraction on driver’s anticipation in a simulated car following scenario. Transportation Research Part F: Traffic Psychology and Behaviour, 14(6), 555–566. https://doi.org/10.1016/j.trf.2011.06.003
NotonD.StarkL. (1971). Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision Research, 11(9), 929-IN8. https://doi.org/10.1016/0042-6989(71)90213-6
29.
R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/
30.
RenkewitzF.JahnG. (2012). Memory indexing: A novel method for tracing memory processes in complex cognitive tasks. Journal of Experimental Psychology: Learning Memory and Cognition, 38(6), 1622–1639. https://doi.org/10.1037/a0028073
31.
RichardsonD. C.SpiveyM. J. (2000). Representation, space and hollywood squares: Looking at things that aren’t there anymore. Cognition, 76(3), 269–295. https://doi.org/10.1016/S0010-0277(00)00084-6
32.
RosnerA.FrankeT.PlattenF.AttigC. (2019). Eye movements in vehicle control. In KleinC.EttingerU. (Eds.), Eye movement research: An introduction to its scientific foundations and applications (pp. 929–969). Springer. https://doi.org/10.1007/978-3-030-20085-5_22
33.
ScholzA.KlichowiczA.KremsJ. F. (2018). Covert shifts of attention can account for the functional role of eye movements to nothing. Memory & Cognition, 46(2), 230–243. https://doi.org/10.3758/s13421-017-0760-x
34.
ScholzA.MehlhornK.BocklischF.KremsJ. (2011). Looking at Nothing Diminishes with practice. In Proceedings of the Annual Meeting of the Cognitive Science Society, 33, 1070–1075.
35.
ScholzA.MehlhornK.KremsJ. F. (2016). Listen up, eye movements play a role in verbal memory retrieval. Psychological Research, 80(1), 149–158. https://doi.org/10.1007/s00426-014-0639-4
SingmannHKellenD (2019). An Introduction to Mixed Models for Experimental Psychology. In SpielerDanielSchumacherEric (Eds.), New Methods in Cognitive Psychology (1st ed., pp. 4–31). New York: Psychology Press. https://doi.org/10.4324/9780429318405.
38.
SivakM. (1996). The information that drivers use: Is it indeed 90% visual?Perception, 25(9), 1081–1089. https://doi.org/10.1068/p251081
39.
SpiveyM. J.GengJ. J. (2001). Oculomotor mechanisms activated by imagery and memory: Eye movements to absent objects. Psychological Research, 65(4), 235–241. https://doi.org/10.1007/s004260100059
40.
VidulichM. A.TsangP. S. (1987). Absolute magnitude estimation and relative judgement approaches to subjective workload assessment. Proceedings of the Human Factors Society Annual Meeting, 31(9), 1057–1061. https://doi.org/10.1177/154193128703100930
41.
WantzA. L.MartarelliC. S.MastF. W. (2016). When looking back to nothing goes back to nothing. Cognitive Processing, 17(1), 105–114. https://doi.org/10.1007/s10339-015-0741-6
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
