The goal of this study was to assess how different real-time gaze sharing visualization techniques affect eye tracking metrics, workload, team situation awareness (TSA), and team performance.
Background
Gaze sharing is a real-time visualization technique that allows teams to know where their team members are looking on a shared display. Gaze sharing visualization techniques are a promising means to improve collaborative performance on simple tasks; however, there needs to be validation of gaze sharing with more complex and dynamic tasks.
Methods
This study evaluated the effect of gaze sharing on eye tracking metrics, workload, team SA, and team performance in a simulated unmanned aerial vehicle (UAV) command-and-control task. Thirty-five teams of two performed UAV tasks under three conditions: one with no gaze sharing and two with gaze sharing. Gaze sharing was presented using a fixation dot (i.e., a translucent colored dot) and a fixation trail (i.e., a trail of the most recent fixations).
Results
The results showed that the fixation trail significantly reduced saccadic activity, lowered workload, supported team SA at all levels, and improved performance compared to no gaze sharing; however, the fixation dot had the opposite effect on performance and SA. In fact, having no gaze sharing outperformed the fixation dot. Participants also preferred the fixation trail for its visibility and ability to track and monitor the history of their partner’s gaze.
Conclusion
The results showed that gaze sharing has the potential to support collaboration, but its effectiveness depends highly on the design and context of use.
Application
The findings suggest that gaze sharing visualization techniques, like the fixation trail, have the potential to improve teamwork in complex UAV tasks and could have broader applicability in a variety of collaborative settings.
AkkilD.JamesJ. M.IsokoskiP.KangasJ. (2016). GazeTorch: Enabling gaze awareness in collaborative physical tasks. In Conference on human factors in computing systems – Proceedings, 07-12-May-2016 (pp. 1151–1158). ACM. https://doi.org/10.1145/2851581.2892459
2.
AkkilD.ThankachanB.IsokoskiP. (2018). I see what you see: Gaze awareness in mobile video collaboration. In Proceedings of the 2018 ACM symposium on eye tracking research & applications (pp. 1–9). ACM. https://doi.org/10.1145/3204493.3204542
3.
AlonsoR.CausseM.VachonF.PariseR.DehaisF.TerrierP. (2013). Evaluation of head-free eye tracking as an input device for air traffic control. Ergonomics, 56(2), 246–255. https://doi.org/10.1080/00140139.2012.744473
4.
AtkinsonR. B.Sidey-GibbonsC.SminkD. S.AskariR.PusicA. L.ChoN. L.RobertsonJ. M.RangelE. L. (2023). Real-time student feedback on the surgical learning environment: Use of a mobile application. Journal of Surgical Education, 80(6), 817–825. https://doi.org/10.1016/j.jsurg.2023.02.017
5.
AtwehJ. A.El IskandaraniM.RiggsS. L. (2024). Real-time gaze sharing techniques and their influence on performance and shared situational awareness of teammates in UAV C2 operations. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 68(1). https://doi.org/10.1177/10711813241260316
6.
AtwehJ. A.HayekJ. A.RiggsS. L. (2023). Quantifying visual attention of teams during workload transitions using aoi-based cross-recurrence metrics. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 67(1), 1882–1887. https://doi.org/10.1177/21695067231193683
7.
AtwehJ. A.HazimehR.RiggsS. L. (2023). Can real-time gaze sharing help team collaboration? A preliminary examination of its effectiveness with Pairs. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 67(1), 716–721. https://doi.org/10.1177/21695067231193659
8.
AtwehJ. A.Marie MoacdiehN.RiggsS. L. (2022). Identifying individual-team-and organizational-level factors that affect team performance in complex domains based on recent literature. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 66(1), 1795–1799. https://doi.org/10.1177/1071181322661213
9.
AtwehJ. A.RiggsS. L. (2024). ScanMatch versus MultiMatch: A comparison of the sensitivity of scanpath similarity metrics to changes in workload. In 2024 systems and information engineering design symposium (SIEDS) (pp. 57–62): IEEE. https://doi.org/10.1109/SIEDS61124.2024.10534720
10.
AtwehJ. A.TabbaraI.NasrallahG.RiggsS. L. (2024). Reducing barriers in analyzing eye tracking data: The development of a GUI to preprocess eye tracking data. In Proceedings of the 2024 systems and information engineering design symposium (SIEDS) (pp. 286–291). IEEE. https://doi.org/10.1109/SIEDS61124.2024.10534671
11.
BergerM.DeyD.DandekarA.BaratiB.PflegingB.BernhauptR. (2023). Empowering driver-passenger collaboration: Designing in-car systems with a focus on social connectedness, fairness, and team performance. International Journal of Human-Computer Interaction, 39(16), 3180–3202. https://doi.org/10.1080/10447318.2023.2205769
12.
BoltonM. L.BiltekoffE.HumphreyL. (2023). The mathematical meaninglessness of the NASA task load index: A level of measurement analysis. IEEE Transactions on Human-Machine Systems, 53(3), 590–599. https://doi.org/10.1109/THMS.2023.3263482
13.
BrennanS. E.ChenX.DickinsonC. A.NeiderM. B.ZelinskyG. J. (2008). Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition, 106(3), 1465–1477. https://doi.org/10.1016/j.cognition.2007.05.012
14.
BylinskiiZ.BorkinM. A.KimN. W.PfisterH.OlivaA. (2015). Eye fixation metrics for large scale evaluation and comparison of information visualizations. In IEEE workshop on eye tracking and visualization. IEEE. https://api.semanticscholar.org/CorpusID:18126165
15.
Cannon-BowersJ. A.SalasE.ConverseS. (1993). Shared mental models in expert team decision making. In CastellanN. J.Jr. (Ed.), Individual and group decision making: Current issues (pp. 221–246). Lawrence Erlbaum Associates, Inc.
16.
ChengS.WangJ.ShenX.ChenY.DeyA. (2022). Collaborative eye tracking-based code review through real-time shared gaze visualization. Frontiers of Computer Science, 16(3), Article 163704. https://doi.org/10.1007/s11704-020-0422-1
17.
D’AngeloS.BegelA. (2017). Improving communication between pair programmers using shared gaze awareness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 6245–6255). ACM. https://doi.org/10.1145/3025453.3025573
18.
DevlinS. P.FlynnJ. R.RiggsS. L. (2019). Examining the visual attention of pairs of operators during a low-to-high workload change. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting. 63(1). 201–205. https://doi.org/10.1177/1071181319631168
19.
de WinterJ. C.EismaY. B.CabrallC. D. D.HancockP. A.StantonN. A. (2019). Situation awareness based on eye movements in relation to the task environment. Cognition, Technology & Work, 21(1), 99–111. https://doi.org/10.1007/s10111-018-0527-6
20.
Di StasiL. L.RennerR.CatenaA.CanasJ. J.VelichkovskyB. M.PannaschS. (2012). Towards a driver fatigue test based on the saccadic main sequence: A partial validation by subjective report data. Transportation Research Part C: Emerging Technologies, 21(1), 122–133. https://doi.org/10.1016/j.trc.2011.07.002
21.
DursoF. T.TruittT. R.HackworthC. A.CrutchfieldJ. M.ManningC. A. (1998). En route operational errors and situational awareness. The International Journal of Aviation Psychology, 8(2), 177–194. https://doi.org/10.1207/s15327108ijap0802_6
22.
El IskandaraniM.AtwehJ. A.McGarryS. P.RiggsS. L.MoacdiehN. M. (2023). Does it multiMatch? What scanpath comparison tells us about task performance in teams. Journal of Cognitive Engineering and Decision Making, 17(3), 294–309. https://doi.org/10.1177/15553434231171484
23.
EndsleyM. R. (1988). Design and evaluation for situation awareness enhancement. Proceedings of the Human Factors Society Annual Meeting, 32(2), 97–101. https://doi.org/10.1177/154193128803200221
24.
EndsleyM. R. (1995a). Direct measurement of situation awareness in simulations of dynamic systems: Validity and use of SAGAT. In Experimental analysis and measurement of situation awareness (pp. 107–113).
EndsleyM. R. (1999). Situation awareness in aviation Systems. In Handbook of aviation human factors (pp. 257–276). Lawrence Erlbaum Associates Publishers.
28.
EndsleyM. R. (2021a). A systematic review and meta-analysis of direct objective measures of situation awareness: A comparison of SAGAT and SPAM. Human Factors: The Journal of the Human Factors and Ergonomics Society, 63(1), 124–150. https://doi.org/10.1177/0018720819875376
29.
EndsleyM. R. (2021b). Situation awareness measurement: How to measure situation awareness in individuals and teams. Human Factors and Ergonomics Society. https://books.google.com/books?id=jgptzgEACAAJ
30.
EndsleyM. R.KaberD. B. (1999). Level of automation effects on performance, situation awareness, and workload in a dynamic control task. Ergonomics, 42(3), 462–492. https://doi.org/10.1080/001401399185595
31.
EndsleyM. R.RodgersM. (1994). Situation awareness information requirements analysis for EN route air traffic control. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 38(1), 71–75. https://doi.org/10.1177/154193129403800113
32.
EntinE. E.HinikerP.GrierR.JeffersonT.VecchioG.HarkinsJ. (2006). Enhancing situational awareness and team performance using network centric technologies. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting. 50(3). 482–486. https://doi.org/10.1177/154193120605000357
33.
FanL.WangJ.LiQ.SongZ.DongJ.BaoF.WangX. (2023). Eye movement characteristics and visual fatigue assessment of virtual reality games with different interaction modes. Frontiers in Neuroscience, 17(1), Article 1173127. https://doi.org/10.3389/fnins.2023.1173127
34.
FasoldF.NicklasA.SeifrizF.SchulK.NoëlB.AschendorfP.KlattS. (2021). Gaze coordination of groups in dynamic events – a tool to facilitate analyses of simultaneous gazes within a team. Frontiers in Psychology, 12(1), Article 656388. https://doi.org/10.3389/fpsyg.2021.656388
35.
FeitshansG.RoweA.DavisJ.HollandM.BergerL. (2008). Vigilant Spirit control station (VSCS): The face of COUNTER. In AIAA guidance, navigation and control conference and exhibit. ARC. https://doi.org/10.2514/6.2008-6309
36.
FindlayJ. M. (2004). Eye scanning and visual search. In HendersonJ. M.FerreiraF. (Eds.), The interface of language, vision, and action: Eye movements and the visual world (pp. 134–159). Psychology Press.
37.
FioreS. M.SalasE. (2004). Why we need team cognition. In SalasE.FioreS. M. (Eds.), Team cognition: Understanding the factors that drive process and performance (pp. 235–248). American Psychological Association. https://doi.org/10.1037/10690-011
GardnerA. K.KosemundM.MartinezJ. (2017). Examining the feasibility and predictive validity of the SAGAT tool to assess situation awareness among medical trainees. Simulation in Healthcare, 12(1), 17–21. https://doi.org/10.1097/SIH.0000000000000181
40.
GormanJ. C.GrimmD. A.StevensR. H.GallowayT.Willemsen-DunlapA. M.HalpinD. J. (2020). Measuring real-time team cognition during team training. Human Factors: The Journal of the Human Factors and Ergonomics Society, 62(5), 825–860. https://doi.org/10.1177/0018720819852791
41.
GuptaK.LeeG. A.BillinghurstM. (2016). Do you see what I see? The effect of gaze tracking on task space remote collaboration. IEEE Transactions on Visualization and Computer Graphics, 22(11), 2413–2422. https://doi.org/10.1109/TVCG.2016.2593778
42.
HarezlakK.KasprowskiP. (2018). Application of eye tracking in medicine: A survey, research issues, and challenges. Computerized Medical Imaging and Graphics, 65(1), 176–190. https://doi.org/10.1016/j.compmedimag.2017.04.006
43.
HartS. G.StavelandL. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52(1), 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9
44.
HeS.ZhiJ.DuY.FengS. (2023). Effects of design features of train electronic guidance interface on visual search, behavior, and usability. International Journal of Industrial Ergonomics, 93(1), Article 103387. https://doi.org/10.1016/j.ergon.2022.103387
45.
HinikerP. J.EntinE. E. (1992). Examining cognitive processes in command crisis: New HEAT experiments on shared battle graphics and time tagging. In Proceedings of the 1992 symposium on command and control research.
46.
HoganM. P.PaceD. E.HapgoodJ.BooneD. C. (2006). Use of human patient simulation and the situation awareness global assessment technique in practical trauma skills assessment. Journal of Trauma and Acute Care Surgery, 61(5), 1047–1052. https://doi.org/10.1097/01.ta.0000238687.23622.89
47.
HoggD. N.TorralbaB.VoldenF. S. (1993). A situation awareness methodology for the evaluation of process control systems: Studies of feasibility and the implications of use. Storefjell, Norway: OECD Halden Reactor Project.
48.
HultinM.JonssonK.HärgestamM.LindkvistM.BrulinC. (2019). Reliability of instruments that measure situation awareness, team performance, and task performance in a simulation setting with medical students. BMJ Open, 9(9), Article e029412. https://doi.org/10.1136/bmjopen-2019-029412
49.
JingA.MayK.MatthewsB.LeeG.BillinghurstM. (2022). The impact of sharing gaze behaviours in collaborative mixed reality. Proceedings of the ACM on Human-Computer Interaction, 6(CSCW2), 1–27. https://doi.org/10.1145/3555564
50.
KapitaniakB.WalczakM.KosobudzkiM.JóźwiakZ.BortkiewiczA. (2015). Application of eye-tracking in the testing of drivers: A review of research. International Journal of Occupational Medicine & Environmental Health, 28(6), 941–954. https://doi.org/10.13075/ijomeh.1896.00317
51.
KhanR. S. A.TienG.AtkinsM. S.ZhengB.PantonO. N. M.MeneghettiA. T. (2012). Analysis of eye gaze: Do novice surgeons look at the same locations as expert surgeons during a laparoscopic operation?Surgical Endoscopy, 26(12), 3536–3540. https://doi.org/10.1007/s00464-012-2400-7
52.
KrauzlisR. J. (2013). Eye movements. In Fundamental neuroscience (pp. 697–714). Academic Press.
53.
KumarA.BurchM.MuellerK. (2018). Visual analysis of eye gazes to assist strategic planning in computer games. In Proceedings of the 3rd workshop on eye tracking and visualization (pp. 1–5). ACM. https://doi.org/10.1145/3205929.3205935
54.
LaskowitzS.GriffinJ. W.GeierC. F.ScherfK. S. (2021). Cracking the code of live human social interactions in autism: A review of the eye-tracking literature. In PalmeroC.Jacques JuniorJ. C. S.ClapésA.GuyonI.TuW.-W.MoeslundT. B.EscaleraS. (Eds.), Understanding social behavior in dyadic and small group interactions (Vol. 173, pp. 242–264). PMLR. https://proceedings.mlr.press/v173/laskowitz22a.html
55.
LeeG. A.KimS.LeeY.DeyA.PiumsomboonT.NormanM.BillinghurstM. (2017). Improving collaboration in augmented video conference using mutually shared gaze. In ICAT-EGVE (pp. 197–204). https://dl.acm.org/doi/abs/10.5555/3298830.3298866
56.
LiF.ChenC. H.XuG.KhooL. P. (2020). Hierarchical eye-tracking data analytics for human fatigue detection at a traffic control center. IEEE Transactions on Human-Machine Systems, 50(5), 465–474. https://doi.org/10.1109/THMS.2020.3016088
57.
LiJ.ManavalanM.D’AngeloS.GergleD. (2016). Designing shared gaze awareness for remote collaboration. In Proceedings of the 19th ACM conference on computer supported cooperative work and social computing companion (pp. 325–328). ACM. https://doi.org/10.1145/2818052.2869097
58.
LiY.LuF.LagesW. S.BowmanD. (2019). Gaze direction visualization techniques for collaborative wide-area model-free augmented reality. In Symposium on spatial user interaction (pp. 1–11). ACM. https://doi.org/10.1145/3357251.3357583
59.
MaoR.LiG.HildreH. P.ZhangH. (2021). A survey of eye tracking in automobile and aviation studies: Implications for eye tracking studies in marine operations. IEEE Transactions on Human-Machine Systems, 51(2), 87–98. https://doi.org/10.1109/THMS.2021.3053196
60.
MatthewsM. D.BealS. A. (2002). Assessing situation awareness in field training exercises. Military Academy West Point NY Office of Military Psychology and Leadership. https://doi.org/10.21236/ADA408560
61.
McCarleyJ. S.LeggettN.EnrightA. (2021). Shared gaze fails to improve team visual monitoring. Human Factors, 63(4), 696–705. https://doi.org/10.1177/0018720820902347
62.
MorganC.ChiappeD.KrautJ.StrybelT. Z.VuK. P. L. (2012). An Investigation into the potential sources of interference in SA probe techniques. In LandryS. (Ed.), Advances in human aspects of aviation (pp. 623–631). CRC Press.
63.
NewnJ.VellosoE.AllisonF.AbdelrahmanY.VetereF. (2017). Evaluating real-time gaze representations to infer intentions in competitive turn-based strategy games. In Proceedings of the annual symposium on computer-human interaction in play (pp. 541–552). ACM. https://doi.org/10.1145/3116595.3116624
64.
NotonD.StarkL. (1971). Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision Research, 11(9), 929–942. https://doi.org/10.1016/0042-6989(71)90213-6
65.
NyströmM.HolmqvistK. (2010). An adaptive algorithm for fixation, saccade, and glissade detection in eye tracking data. Behavior Research Methods, 42(1), 188–204. https://doi.org/10.3758/BRM.42.1.188
66.
PooleA.BallL. J. (2006). Eye tracking in HCI and usability research. In Encyclopedia of human computer interaction (pp. 211–219). IGI global.
67.
Reinerman-JonesL. E.HughesN.D’AgostinoA.MatthewsG. (2019). Human performance metrics for the nuclear domain: A tool for evaluating measures of workload, situation awareness, and teamwork. International Journal of Industrial Ergonomics, 69(1), 217–227. https://doi.org/10.1016/j.ergon.2018.12.001
68.
RogersS. L.SpeelmanC. P.GuidettiO.LongmuirM. (2018). Using dual eye tracking to uncover personal gaze patterns during social interaction. Scientific Reports, 8(1), 4271. https://doi.org/10.1038/s41598-018-22726-7
69.
SchneiderB.PeaR. (2017). Real-time mutual gaze perception enhances collaborative learning and collaboration quality. In OreyM.BranchR. (Eds.), Educational media and technology yearbook (Vol. 40, pp. 99–125). Springer. https://doi.org/10.1007/978-3-319-45001-8_7
70.
ShreeD. V. J.MurthyL. R. D.SalujaK. S.BiswasP. (2018). Operating different displays in military fast jets using eye gaze tracker. Journal of Aviation Technology and Engineering, 8(1), 31. https://doi.org/10.7771/2159-6670.1184
71.
SkaramagkasV.GiannakakisG.KtistakisE.ManousosD.KaratzanisI.TachosN. S.TripolitiE.MariasK.FotiadisD. I.TsiknakisM. (2021). Review of eye tracking metrics involved in emotional and cognitive processes. IEEE Reviews in Biomedical Engineering, 16(1), 260–277. https://doi.org/10.1109/RBME.2021.3066072
72.
ŠkvarekováI.PechoP.AžaltovičV.KanderaB. (2020). Number of saccades and fixation duration as indicators of pilot workload. Transportation Research Procedia, 51(1), 67–74. https://doi.org/10.1016/j.trpro.2020.11.009
SteinR.BrennanS. E. (2004). Another person’s eye gaze as a cue in solving programming problems. In Proceedings of the 6th international conference on multimodal interfaces - ICMI ’04 (Vol. 9). ACM. https://doi.org/10.1145/1027933.1027936
75.
SulistyawatiK.ChuiY. P.WickensC. D. (2008). Multi-method approach to team situation awareness. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting. 52(4). 463–467. https://doi.org/10.1177/154193120805200455
76.
SungG.FengT.SchneiderB. (2021). Learners learn more and instructors track better with real-time gaze sharing. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1–23. https://doi.org/10.1145/3449208
77.
TaylorR. M. (1990). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. In Situational awareness (pp. 111–128). https://doi.org/10.4324/9781315087924-8
78.
TröstererS.GärtnerM.WuchseM.MaurerB.BaumgartnerA.MeschtscherjakovA.TscheligiM. (2015). Four eyes see more than two: Shared gaze in the car. In IFIP conference on human-computer interaction (pp. 331–348). https://doi.org/10.1007/978-3-319-22668-2_26
79.
TsaiY. F.ViirreE.StrychaczC.ChaseB.JungT. P. (2007). Task performance and eye activity: Predicting behavior relating to cognitive workload. Aviation Space & Environmental Medicine, 78(5 Suppl), B176–B185.
80.
VesperC.SchmitzL.SafraL.SebanzN.KnoblichG. (2016). The role of shared visual information for joint action coordination. Cognition, 153(1), 118–123. https://doi.org/10.1016/j.cognition.2016.05.002
81.
WaltonJ.GilbertS. B. (2022). Evaluating the effect of displaying team vs. individual metrics on team performance. International Journal of Human-Computer Studies, 160(4), Article 102759. https://doi.org/10.1016/j.ijhcs.2021.102759
82.
WangQ.YangY.WangQ.MaQ. (2014). The effect of human image in B2C website design: An eye-tracking study. Enterprise Information Systems, 8(5), 582–605. https://doi.org/10.1080/17517575.2014.925585
83.
YangQ.WangT.SuN.XiaoS.KapoulaZ. (2013). Specific saccade deficits in patients with Alzheimer’s disease at mild to moderate stage and in patients with amnestic mild cognitive impairment. Age, 35(4), 1287–1298. https://doi.org/10.1007/s11357-012-9420-z
84.
YarbusA. L. (1967). Eye movements during perception of complex objects. In YarbusA. L. (Ed.), Eye movements and vision (pp. 171–211). Springer US. https://doi.org/10.1007/978-1-4899-5379-7_8
85.
ZhangY.PfeufferK.ChongM. K.AlexanderJ.BullingA.GellersenH. (2017). Look together: Using gaze for assisting co-located collaborative search. Personal and Ubiquitous Computing, 21(1), 173–186. https://doi.org/10.1007/s00779-016-0969-x
