Eye Activity Correlates of Workload during a Visuospatial Memory Task

Abstract

Changes in six measures of eye activity were assessed as a function of task workload in a target identification memory task. Eleven participants completed four 2hr blocks of a mock anti-air-warfare task, in which they were required to examine and remember target classifications (friend/enemy) for subsequent prosecution (fire upon/allow to pass), while targets moved steadily toward two centrally located ship icons. Target density served as the task workload variable; between one and nine targets were simultaneously present on the display. For each participant, moving estimates of blink frequency and duration, fixation frequency and dwell time, saccadic extent, and mean pupil diameter, integrated over periods of 10 to 20 s, demonstrated systematic changes as a function of target density. Nonlinear regression analyses found blink frequency, fixation frequency, and pupil diameter to be the most predictive variables relating eye activity to target density Participant-specific artificial neural network models, developed through training on two or three sessions and subsequently tested on a different session from the same participant, correlated well with actual target density levels (mean R = 0.66). Results indicate that moving mean estimation and artificial neural network techniques enable information from multiple eye measures to be combined to produce reliable near-real-time indicators of workload in some visuospatial tasks. Potential applications include the monitoring of visual activity of system operators for indications of visual workload and scanning efficiency.

Get full access to this article

View all access options for this article.

References

Ackerman, P. L. , & Kanfer, R. (1994). Improving problem-solving and decision-making skills under stress: Prediction and training (Final Report for Contract N00014-91-J-4159). Minneapolis: University of Minnesota.

Backs, R. W. , & Walrath, L. C. (1992). Eye movements and pupillary response indices of mental workload during visual search of symbolic displays. Applied Ergonomics, 23, 243–254.

Beatty, J. & Wagoner, B. L. (1978). Pupillometric signs of brain activation vary with level of cognitive processing. Science, 199, 1216–1218.

Brookings, J. B. , Wilson, G. F. , & Swain, C. R. (1996). Psycho-physiological responses to changes in workload during simulated air traffic control. Biological Psychology, 42, 361–377.

Byrne, E. A. , & Parasuraman, R. (1996). Psychophysiology and adaptive automation. Biological Psychology, 42, 249–268.

Callan, D. J. (1998). Eye movement relationships to excessive performance error in aviation. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting (pp. 1132–1136). Santa Monica, CA: Human Factors and Ergonomics Society.

Fogarty, C. & Stern, J. A. (1989). Eye movements and blinks: Their relationship to higher cognitive processes. International Journal of Psychophysiology, 8, 35–42.

Gevins, A. Smith, M. E. , Leong, H. , McEvoy, L. , Whitfield, S. , Du, R. , & Rush, G. (1998). Monitoring working memory load during computer-based tasks with EEG pattern recognition methods. Human Factors, 40, 79–91.

Granholm, E. Asarnow, R. F. , Sarkin, A. J. , & Dykes, K. L. (1996). Pupillary responses index cognitive resource limitations. Psychophysiology, 3, 457–461.

10.

Hankins, T. C. , & Wilson, G. F. (1998). A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. Aviation, Space and Environmental Medicine, 69, 360–367.

11.

Itoh, Y. Hayashi, Y. , Tsukui, I. , & Saito, S. (1990). The ergonomic evaluation of eye movements and mental workload in aircraft pilots. Ergonomics, 33, 719–733.

12.

Jung, T.-P. , Makeig, S. , Stensmo, M. , & Sejnowski, T. J. (1997). Estimating alertness from the EEG power spectrum. IEEE Transactions on Biomedical Engineering, 44, 60–69.

13.

Just, M. A. , & Carpenter, P. A. (1993). The intensity dimension of thought: Pupillometric indices of sentence processing. Canadian Journal of Experimental Psychology, 47, 310–339.

14.

Katoh, Z. (1997). Saccade amplitude as a discriminator of flight types. Aviation, Space and Environmental Medicine, 68, 205–208.

15.

Kramer, A. F. , Trejo, L. J. , & Humphrey, D. G. (1996). Psycho-physiological measures of workload: Potential applications to adaptively automated systems. In R. Parasuraman & M. Mouloua (Eds.), Automation and human performance (pp. 137–162). Mahwah, NJ: Erlbaum.

16.

Makeig, S. & Inlow, M. (1993). Lapses in alertness: Coherence of fluctuations in performance and the EEG spectrum. Electro-encephalography and Clinical Neurophysiology, 86, 23–35.

17.

Parasuraman, R. Bahri, T. , Deaton, J. E. , Morrison, J. G. , & Barnes, M. (1992). Theory and design of adaptive automation in aviation systems (Progress Report No. NAWCADWAR-92033-60). Warminster, PA: Naval Air Warfare Center.

18.

Pastoor, S. Liu, J. , & Renault, S. (1999). An experimental multimedia system allowing 3-D visualization and eye-controlled interaction without user-worn devices. IEEE Transactions on Multimedia, 1, 41–52.

19.

Van Orden, K. F. , Jung, T.-P. , & Makeig, S. (2000). Combined eye activity measures accurately estimate changes in sustained visual task performance. Biological Psychology, 52, 221–240.

20.

Van Orden, K. F. , Nugent, W. , Lafleur, B. , & Moncho, S. (1999). Assessment of variable coded symbology using visual search performance and eye fixation measures (NHRC Technical Report No. 99-4). San Diego, CA: Naval Health Research Center.

21.

Veltman, J. A. , & Gaillard, A. W. K. (1998). Physiological workload reactions to increasing levels of task difficulty. Ergonomics, 41, 656–669.

22.

Wilson, G. F. , Fullenkamp, B. S. , & Davis, I. (1994). Evoked potential, cardiac, blink, and respiration measures of pilot workload in air-to-ground missions. Aviation, Space and Environmental Medicine, 65, 100–105.

23.

Zelinsky, G. J. , Rajesh, P. N. R. , Hayhoe, M. M. , & Ballard, D. H. (1997). Eye movements reveal the spatiotemporal dynamics of visual search. Psychological Science, 8, 448–453.