Studies investigating the effects of level of autonomy (LOA) on workload and performance in human-agent teams typically utilize subjective measures, but do not often incorporate physiological measures. This paper examines how well eye movement data collected in a recent experiment converges with the findings suggested by the subjective measures. Several eye behavior measures (fixation count, average fixation duration, blink rate, saccade amplitude and pupil diameter) were evaluated, and findings based on these compared to findings of NASA-TLX and situation awareness questionnaires. In addition, individual differences due to perceived attentional control were evaluated. Findings indicate that the physiological measures account for variance in workload that typical subjective measures may not, as they are more sensitive to individual differences among participants.
Get full access to this article
View all access options for this article.
References
1.
BarberD.DavisL.NicholsonD.FinkelsteinNChenJ. Y. C. (2008). The mixed initiative experimental (MIX) testbed for human robot interactions with varied levels of automation. In Proceedings of the 26th Army Science Conference (Orlando, FL, Dec. 1-4, 2008). ASC ’08. US Dept. of Army, Washington, DC.
2.
ChenJ. Y. C.BarnesM. J. (2014). Human–agent teaming for multirobot control: A review of human factors issues. IEEE Transactions on Human-Machine Systems, 44(1), 13-29.
3.
ChenJ. Y. C.BarnesM. J. (2012a). Supervisory control of multiple robots: Effects of imperfect automation and individual differences. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(2), 157-174.
4.
ChenJ.Y.C.BarnesM.J. (2012b). Supervisory control of multiple robots in dynamic tasking environments. Ergonomics, 55, 1043 – 1058.
5.
ChenJ. Y.BarnesM. J.Harper-SciariniM. (2011). Supervisory control of multiple robots: Human-performance issues and user-interface design. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 41(4), 435-454.
6.
ChenJ. Y. C.JoynerC. T. (2009). Concurrent performance of gunner’s and robotic operator’s tasks in a multi-tasking environment. Military Psychology, 21(1), 98-113.
7.
DerryberryD.ReedM. A. (2002). Anxiety-related attentional biases and their regulation by attentional control. Journal of Abnormal Psychology, 111(2), 225-236.
8.
DrexlerJ.M.ChenJ.Y.C.QuinnS.A.SolomonD. (2012). Effects of display dimension and viewpoint synchronization on robot teleoperation performance in a simulated environment. (ARL-TR-6255). Aberdeen Proving Ground, MD: U.S Army Research Laboratory.
9.
EhmkeC.WilsonS. (2007, September). Identifying web usability problems from eye-tracking data. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI… but not as we know it-Volume 1, 119-128. British Computer Society.
10.
GoldbergJ. H.KotvalX. P. (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24(6), 631-645.
11.
GoodrichM. A.QuigleyM.CosenzoK. (2005). Task switching and multi-robot teams. In Multi-Robot Systems. From Swarms to Intelligent Automata Volume III (pp. 185-195). Springer Netherlands.
12.
HartS.StavelandL. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In HancockP.MeshkatiN. (Eds.), Human mental workload (pp. 139-183). Amsterdam: Elsevier.
13.
HoffmanJ. E. (1998). Visual attention and eye movements. In PashlerH. (Ed.), Attention (pp. 119-153). Hove England: Psychology Press/Erlbaum (UK) Taylor & Francis.
14.
IqbalS. T.ZhengX. S.BaileyB. P. (2004, April). Task-evoked pupillary response to mental workload in human-computer interaction. In CHI’04 extended abstracts on Human factors in computing systems, 1477-1480.
15.
JacobR. J.KarnK. S. (2003). Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind, 2(3), 4.
16.
KangM. J.HsuM.KrajbichI. M.LoewensteinG.McClureS. M.WangJ. T. Y.CamererC. F. (2009). The wick in the candle of learning: Epistemic curiosity activates reward circuitry and enhances memory. Psychological Science, 20(8), 963-973.
17.
KasarskisP.StehwienJ.HickoxJ.AretzA.WickensC. (2001). Comparison of expert and novice scan behaviors during VFR flight. In Proceedings of the 11th International Symposium on Aviation Psychology.
18.
MayJ. G.KennedyR. S.WilliamsM. C.DunlapW. P.BrannanJ. R. (1990). Eye movement indices of mental workload. Acta psychologica, 75(1), 75.
19.
PeavlerW. S. (1974). Pupil size, information overload, and performance differences. Psychophysiology11 (5), 559–566.
20.
RecarteM. A.NunesL. M. (2003). Mental workload while driving: effects on visual search, discrimination, and decision making. Journal of Experimental Psychology: Applied; Journal of Experimental Psychology: Applied, 9(2), 119.
21.
SnyderM.QuZ.ChenJ. Y. C.BarnesM. (2010). RoboLeader for reconnaissance by a team of robotic vehicles. In Proceedings of the International Symposium on Collaborative Technologies and Systems (Chicago, May 17-21, 2010). CTS ’10. IEEE, New York, 522-530.
22.
SternJ. A.BoyerD.SchroederD. (1994). Blink rate: a possible measure of fatigue. Human Factors: The Journal of the Human Factors and Ergonomics Society, 36(2), 285-297.
Van OrdenK. F.LimbertW.MakeigS.JungT. P. (2001). Eye activity correlates of workload during a visuospatial memory task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 43(1), 111-121.
25.
WrightJ.L.ChenJ.Y.C.QuinnS.A.BarnesM.J. (2013). The effects of level of autonomy on human-agent teaming for multi-robot control and local security maintenance, Tech. Rep. (ARL-TR-6724), U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA.
26.
ZelinskyG. J.SheinbergD. L. (1997). Eye movements during parallel–serial visual search. Journal of Experimental Psychology: Human Perception and Performance, 23(1), 244.
