Interaction cues, which inform users about potential actions to take, are common to many types of extended reality applications. While numerous studies have compared individual interaction cues, few studies have investigated combinations of interaction cues for complex tasks, particularly ones involving high counts of similar actions. We present a within-subject study (n = 48) investigating the effects of interaction cue combinations for guiding users through in-cockpit procedures for a UH-60 Black Hawk helicopter. The results of our study indicate significant effects for different interaction cue combinations for perceived mental effort and discomfort. Furthermore, the completion time results in our study contradict previous results pertaining to the effectiveness of specific interaction cues, which we believe are due to the context and complexity of the real-world application underlying our study. These results imply that Look Arrow and Pick Arrow are the optimal interaction cue combination for scenarios that involve dense operating environments.
BioccaF.TangA.OwenC.XiaoF. (2006, April). Attention funnel: omnidirectional 3D cursor for mobile augmented reality platforms. In Proceedings of the SIGCHI conference on Human Factors in computing systems (pp. 1115–1122).
2.
BorkF.SchnelzerC.EckU.NavabN. (2018). Towards efficient visual guidance in limited field-of-view head-mounted displays. IEEE Transactions on Visualization and Computer Graphics, 24(11), 2983–2992.
3.
BowmanD. A.McMahanR. P. (2007). Virtual reality: How much immersion is enough?Computer, 40(7), 36–43.
4.
BridgemanR.NevilleK. J.MasseyL.KrauskopfC.MizanA.MooneyJ.SchmorrowD. (2019). Human factors electronic kneeboard design guidelines for military tactical aviation. In 20th International Symposium on Aviation Psychology (p. 343).
5.
DillmanK. R.MokT. T. H.TangA.OehlbergL.MitchellA. (2018, April). A visual interaction cue framework from video game environments for augmented reality. In Proceedings of the 2018 CHI conference on human factors in computing systems (pp. 1–12), Montreal, QC, Canada.
6.
FernandesA. S.FeinerS. K. (2016, March). Combating VR sickness through subtle dynamic field-of-view modification. In 2016 IEEE symposium on 3D user interfaces (3DUI) (pp. 201–210). IEEE.
7.
GattulloM.EvangelistaA.UvaA. E.FiorentinoM.GabbardJ. L. (2020). What, how, and why are visual assets used in industrial augmented reality? A systematic review and classification in maintenance, assembly, and training (from 1997 to 2019). IEEE transactions on visualization and computer graphics, 28(2), 1443–1456.
8.
GruenefeldU.EnnengaD.AliA. E.HeutenW.BollS. (2017, October). Eyesee360: Designing a visualization technique for out-of-view objects in head-mounted augmented reality. In Proceedings of the 5th symposium on spatial user interaction (pp. 109–118).
9.
GruenefeldU.LangeD.HammerL.BollS.HeutenW. (2018, June). Flyingarrow: Pointing towards out-of-view objects on augmented reality devices. In Proceedings of the 7th ACM international symposium on pervasive displays (pp. 1–6).
10.
GruenefeldU.PrädelL.HeutenW. (2019, November). Locating nearby physical objects in augmented reality. In Proceedings of the 18th international conference on mobile and ubiquitous multimedia (pp. 1–10).
11.
HaradaY.OhyamaJ. (2022). Quantitative evaluation of visual guidance effects for 360-degree directions. Virtual Reality, 26(2), 759–770.
12.
HendersonS. J.FeinerS. K. (2011, October). Augmented reality in the psychomotor phase of a procedural task. In 2011 10th IEEE international symposium on mixed and augmented reality (pp. 191–200). IEEE.
13.
HowellM. J.HerreraN. S.MooreA. G.McMahanR. P. (2016). A reproducible olfactory display for exploring olfaction in immersive media experiences. Multimedia Tools and Applications, 75, 12311–12330.
14.
HuX.MooreA.Coleman EubanksJ.AiyazA.,& P.McMahanR. (2020, October). Evaluating interaction cue purpose and timing for learning and retaining virtual reality training. In Proceedings of the 2020 ACM symposium on spatial user interaction (pp. 1–9).
15.
JungJ.LeeH.ChoiJ.NandaA.GruenefeldU.StratmannT.HeutenW. (2018, October). Ensuring safety in augmented reality from trade-off between immersion and situation awareness. In 2018 IEEE international symposium on mixed and augmented reality (ISMAR) (pp. 70–79). IEEE.
16.
KimS.LeeG.HuangW.KimH.WooW.BillinghurstM. (2019, May). Evaluating the combination of visual communication cues for HMD-based mixed reality remote collaboration. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–13).
17.
KumaranR.KimY. J.MilnerA. E.BullockT.GiesbrechtB.HöllererT. (2023, April). The impact of navigation aids on search performance and object recall in wide-area augmented reality. In Proceedings of the 2023 CHI conference on human factors in computing systems (pp. 1–17).
18.
LangeD.StratmannT. C.GruenefeldU.BollS. (2020, April). Hivefive: Immersion preserving attention guidance in virtual reality. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1–13).
19.
LaviolaE.GattulloM.EvangelistaA.FiorentinoM.UvaA. E. (2023). In-situ or side-by-side? A user study on augmented reality maintenance instructions in blind areas. Computers in Industry, 144, 103795.
20.
LaViolaJ. J.JrKruijffE.McMahanR. P.BowmanD.PoupyrevI. P. (2017). 3D user interfaces: Theory and practice. Addison-Wesley Professional.
21.
LeeH.KimH.MonteiroD. V.GohY.HanD.LiangH. N.JungJ. (2019, October). Annotation vs. Virtual tutor: Comparative analysis on the effectiveness of visual instructions in immersive virtual reality. In 2019 IEEE International symposium on mixed and augmented reality (ISMAR) (pp. 318-327). IEEE.
22.
LiuJ. S.ElvezioC.TverskyB.FeinerS. (2021). Using multi-level precueing to improve performance in path-following tasks in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 27(11), 4311–4320.
23.
LokkiT.GrohnM. (2005). Navigation with auditory cues in a virtual environment. IEEE Multimedia, 12(2), 80–86.
24.
MarquardtA.TrepkowskiC.EibichT. D.MaieroJ.KruijffE.SchoningJ. (2020). Comparing non-visual and visual guidance methods for narrow field of view augmented reality displays. IEEE Transactions on Visualization and Computer Graphics, 26(12), 3389–3401.
25.
MuresanA.McintoshJ.HornbækK. (2023). Using feedforward to reveal interaction possibilities in virtual reality. ACM Transactions on Computer Human Interaction, 30(6), 1–47.
26.
PaasF. G. W. C.Van MerriënboerJ. J. G. (1993). The efficiency of instructional conditions: An approach to combine mental effort and performance measures. Human Factors, 35(4), 737–743.
27.
RennerP.PfeifferT. (2017, March). Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. In 2017 IEEE symposium on 3D user interfaces (3DUI) (pp. 186–194). IEEE.
28.
RotheS.BuschekD.HußmannH. (2019). Guidance in cinematic virtual reality-taxonomy, research status and challenges. Multimodal Technologies and Interaction, 3(1), 19.
29.
SchwerdtfegerB.KlinkerG. (2008, September). Supporting order picking with augmented reality. In 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (pp. 91–94). IEEE.
30.
SchwerdtfegerB.ReifR.GünthnerW. A.KlinkerG. (2011). Pick-by-vision: there is something to pick at the end of the augmented tunnel. Virtual Reality, 15, 213–223.
31.
SeeligerA.MerzG.HolzC.FeuerriegelS. (2021, October). Exploring the effect of visual cues on eye gaze during AR-guided picking and assembly tasks. In 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) (pp. 159–164). IEEE.
32.
WallgrünJ. O.BagherM. M.SajjadiP.KlippelA. (2020, March). A comparison of visual attention guiding approaches for 360 image-based vr tours. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 83–91). IEEE.
33.
WickensC. D. (2024). The multiple resource theory and model. Some misconceptions in data interpretations. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 68(1), 713–717.
34.
YuD.LiangH. N.FanK.ZhangH.FlemingC.PapangelisK. (2019). Design and evaluation of visualization techniques of off-screen and occluded targets in virtual reality environments. IEEE transactions on visualization and computer graphics, 26(9), 2762–2774.
