Sage Journals: Discover world-class research

Abstract

Visual search tasks occur in real-world situations, like soldiers in the battlefield identifying hostile targets, where information may fall outside the immediate field of view (FOV). This experiment assessed the effectiveness of attention cues in a wide FOV search task, comparing single versus dual cues that were either perfectly or imperfectly reliable. Participants searched for routine targets and an occasional high-priority target. Results showed that perfect cues improved performance more than an unaided search, with dual cues outperforming single cues. However, when cues were imperfect and automation failed, performance suffered. The notable increase in errors during automation failures suggests a level of automation bias, which was greatest with the dual cues. These findings highlight the benefits of dual cues in wide FOV scenes where critical information may be outside the immediate FOV. However, in high-stakes situations, like detecting explosive devices, caution is advised when cues are imperfect.

Keywords

visual search attention guidance display design perception automation

Introduction

Visual search occurs often in everyday life, such as looking for a folder on a computer or an application on a smartphone. While there are few costs when these routine searches are long or fail, prolonged or failed searches in critical real-world situations can have serious consequences. For instance, a doctor who misses a tumor on an X-ray may give a wrong diagnosis (Wu & Wolfe, 2019), or if a transportation security officer (TSO) misses a weapon in luggage during baggage screening there may be a security threat (Wiegmann et al., 2006). Searching for critical targets without an automation-based aid to guide attention can result in time-consuming and inaccurate searches (Wolfe, 2021). In these safety-critical and time-sensitive situations, attention cueing is beneficial (Posner, 1980; Wickens et al., 2022; Wolfe, 2021; Yeh & Wickens, 2001). While these searches are usually conducted on 2D displays with a small field-of-view (FOV), real-life searches often occur within larger FOVs, where targets may fall outside the immediate FOV where eye scanning or head movements are required to see information (Warden et al., 2024). For example, consider a soldier on the battlefield searching for an explosive device as they traverse enemy terrain, where guiding attention to information outside the immediate FOV as quickly and accurately as possible is critical.

Visual attention cues can provide either identity or spatial information about the target’s location (Warden et al., 2023). Identity cues, like an icon, continuously show the target’s exact appearance during the search task. Spatial cues can provide general or specific information about a target’s location, which may influence whether attention is distributed broadly or narrowly (LaBerge & Brown, 1989). Cues that convey global spatial information provide the general direction of a target’s location, like a minimap showing a birds-eye perspective of the search area with the relative location of a target or an arrow pointing to the left or right of the current gaze. Cues that convey local spatial information provide the target’s specific location, like a highlight cue that outlines the target in the search scene.

Previous studies have shown that spatial cues aided search performance more than identity cues, especially when the cues were perfectly reliable (Warden et al., 2023). But when the cues erred, performance was worse due to an automation bias where people blindly followed the automation recommendation without checking the raw data (Warden et al., 2023). These two studies examined spatial cues that provided global spatial information, but it remains unclear how different levels of spatial cue precision (i.e., global vs. local) impact search performance.

Few studies have explicitly compared global versus local spatial cues, and how these cues differentially influence cue effectiveness in wide FOV searches. Furthermore, there is a lack of research examining the efficacy of dual cueing—presenting two visual cues simultaneously (e.g., global plus local spatial cues or identity plus local spatial cue)—particularly within the context of a wide FOV search task using imperfectly reliable cues that err on occasion. Most cueing studies have focused on the effectiveness of single cues when directing attention to a single target (Warden et al., 2023; Yeh & Wickens, 2001).

In fact, most studies examining multiple cueing have done so in the context of simultaneous multisensory cues, such as a single visual cue presented simultaneously with a single auditory cue (Mahoney et al., 2012; Santangelo et al., 2008). One study found a search performance benefit for a combined visual and auditory cue compared to a visual cue only (Binetti et al., 2021). However, using an auditory cue is not always feasible due to safety reasons or loud environments.

Another study directly assessed using two visual cues during a visual search task completed with an augmented reality (AR) head-mounted display (HMD), finding that search performance was better with two cues than with a single cue (Hein et al., 2020). However, the study failed to include an unaided search condition, which is necessary to assess the relative benefit of cueing compared to an aided search. Additionally, the relative benefit of two versus one cue was inconsistent across the cue combinations, and the cues in the experiment always identified the target correctly (i.e., were not imperfect).

The current research addresses two primary questions: (1) how effective is attention cueing in a wide FOV search task, when the target may not be visible in the initial view of the scene where head movements are needed to locate the target, and (2) how is cue effectiveness moderated by the number of cues (i.e., dual vs. single cues) presented and by the cue reliability (i.e., perfect vs. imperfect cues). Such imperfection may result when automation is asked to make a rapid real-time identification of a suspected threat, using a machine learning algorithm (Raikwar et al., 2024). Hypothesis 1A (H_1A), predicted that cueing would be more effective than no cueing, but there would be a performance cost for imperfect cues resulting in an automaton bias (H_1B). Hypothesis 2 (H₂) predicted that dual cues would improve search performance more than single cues for objects located outside the immediate FOV. Hypothesis 3A (H_3A) predicted that dual cueing would be superior to single cueing when automation is perfect but when automation is imperfect dual cues will actually amplify the automation bias (H_3B).

Method

Participants

Thirty participants were recruited from the psychology research pool at Colorado State University and received course credit in exchange for completing the experiment. All participants had normal or corrected-to-normal vision and were screened for colorblindness. The Institutional Review Board approved the experiment.

Stimuli and Apparatus

Participants completed the experiment using a Samsung 49″ Odyssey Neo G9 Gaming DQHD Quantum Mini-LET wide-angle 2D desktop monitor. The experiment was developed in the game engine Unity (Version 2022.1.21f1). The search scene consisted of naturalistic flat terrain images with little foliage. Embedded in the scene were routine objects (logs, rocks, plastic bottles, and cans) and an occasional high priority object (see Figure 1).

Figure 1.

An illustration of an uncued trial displayed on a wide-angle monitor.

The high priority target was a landmine, simulating an improvised explosive device (IED) search that a soldier might conduct. A total of 128 unique routine objects and five unique high priority objects were available for the search task. Thirty-seven objects were uniformly distributed across the search scene for each trial. The contrast and brightness of all objects matched the search scene. To simulate pictorial depth, the objects apparent size was approximated based on whether the object was located in the foreground, middle ground, or background of the search scene.

Sixteen cue conditions were created, including the no-cue condition (i.e., the unaided search). The attention cues provided either global spatial information (global cue), local spatial information (local cue), or identity information (icon cue). Global cues indicated the general direction of the target’s location. For example, the global arrow cue was always presented at the center of the display and pointed in the direction of the target. Local cues indicated the exact location of the target, such as a yellow highlight cue outlining the target in the search scene. The identity (icon) cue continuously displayed the target’s appearance at the center of the display.

Participants used either single cues or dual cues (Figure 2). Dual cues always contained a local cue and consisted of either a global and local cue or an icon and local cue.

Figure 2.

An illustration of each single cue indicated with the cue name and acronym (a) and each dual-cue (b) condition.

The attention cues were either perfect or imperfect. Perfect cues correctly cued the target 100% of the time. Imperfect cues failed to cue the target 25% of the time (i.e., miss rate). In the imperfect condition, the automation cued a location where no target was present, representing a miss. Such an error might occur when the sensors on an HMD worn by a soldier encounter interference causing them to cue a location where no enemy vehicle is present. In the experiment, the correct target was always present and located anywhere within the scene relative to the empty cued location. For the dual-cue condition, both cues exhibited the same error. All cueing conditions exclusively cued the routine target only. The high priority target was never cued.

Design

The experiment was a 16 (cue type) × 2 (cue reliability) within-subjects repeated measures design. Participants completed two blocks counterbalanced by reliability. Within each reliability block, participants completed 16 cue conditions, which were counterbalanced within the cue reliability blocks. Each cue condition consisted of eight randomized trials. There were 128 trials for each reliability block for a total of 256 trials. Participants completed two practice trials for each cue before the experimental trials. The experiment was 1 hr.

Task

The experiment consisted of a 128-degree static visual search task completed on a 2D desktop display. Participants searched for a routine target that was cued or uncued and an occasional, uncued high priority target. Participants were told that the routine target appeared on every trial and that the high priority object would appear on 13% of the trials, but participants were instructed that the high priority target should take precedence over the routine target. For the imperfect cues, participants were told that the cueing aids may not be perfect.

Procedure

Before the experiment, participants gave consent after reading and signing the consent documentation Then participants completed an electronic colorblindness test and read the instructions for the experiment. Participants sat 15.5 inches from the monitor to achieve a visual angle of 128 degrees. Participants completed practice trials with feedback before the experimental trials. Responses were made using a Logitech M795 Marathon wireless mouse. After selecting the target(s), participants pressed the left “CTRL” button on the keyboard to continue to the next trial.

Results

Before the analysis, data were examined for outliers using the z-score method for both response time and accuracy. One participant was excluded from the analysis. All remaining data (N = 30) were analyzed in R Studio. The assumption of normality was violated (Shapiro-Wilk normality test, ps < .05). All reported ANOVA analyses include the Greenhouse-Geisser (GG) correction to account for the violation. In 3% of the trials, participants made no object selection; therefore, these trials were excluded from all analyses.

Cue Effectiveness for Perfect Single Cue Conditions

Cue effectiveness was examined for the single, perfect cue conditions and the no-cue condition: the icon, minimap, global arrow, local arrow, highlight, gaze guidance line, and the no-cue. A one-way repeated measures ANOVA was used to examine the overall effect of cue type for both response time and percent error. Table 1 reports the mean response time and percent error for the cue conditions stated above.

Table 1.

Mean Response Time (Seconds, s) and Percent Error for the No-Cue and the Single, Perfect Cue Conditions.

Cue condition	Response time (s)	Percent error (%)
Icon	5.6 (0.59)	5.2 (3.0)
Minimap	3.7 (0.45)	6.0 (3.0)
Global arrow	3.3 (0.44)	14.5 (4.6)
Local arrow	2.1 (0.16)	0.0 (0.0)
Highlight	2.1 (0.20)	0.88 (1.2)
Gaze guidance line	1.7 (0.11)	0.0 (0.0)
No cue	6.5 (0.68)	20.9 (5.3)

Note. The 95% CI is reported in the parentheses.

Response Time

The ANOVA revealed that response time was significantly reduced with all cues compared to the no-cue, F(3.37, 90.88) = 64.19, p < .001, $η_{p}^{2}$ = 0.70.

Percent Error

The second ANOVA revealed that all cues significantly reduced errors more than the no-cue, F(6, 162) = 16.41, p < .001, $η_{p}^{2}$ = 0.38.

Effect of Dual Cueing

A 2 (cue type) × 2 (FOV) repeated measures ANOVA was conducted to examine the effect of cue type as a function of whether the target was located in or out of the immediate FOV. Cue type was categorized as single or dual cues. Targets located beyond 25 degrees of visual angle from the center of the display were considered as outside the immediate FOV.

Response Time

Figure 3 shows mean response time.

Figure 3.

Mean response time (seconds) for cue type (single, dual) and target object location relative to the immediate field of view (blue filled bars = in FOV; unfilled red bars = out of FOV).

The ANOVA revealed a significant 0.92 second (s) advantage of dual cueing on response time, F(1, 29) = 97.75, p < .001, $η_{p}^{2}$ = 0.77, The effect of FOV was also significant, F(1, 29) = 27.96, p < .001, $η_{p}^{2}$ = 0.49, showing greater response times when the target was outside of the FOV (M = 3.27 s) compared to inside (M = 2.68 s). The interaction approached conventional levels of significance, F(1, 29) = 3.62, p = .07, $η_{p}^{2}$ = 0.11. The pattern of results suggests searches are slower for both single and dual cues when the target is located outside of the FOV, but dual cues reduce search time more relative to single cues (t(29) = 8.81, p < .001, 95% CI [0.78, 1.25], d = 1.25.

Percent Error

The percent error data are shown in Figure 4. Like response time, dual cues significantly reduced errors by 2.5% compared to single cues, F(1, 29) = 9.52, p = .004, $η_{p}^{2}$ = 0.25. There was a marginally significant effect of FOV, F(1, 29) = 3.10, p = .09, $η_{p}^{2}$ = 0.11, suggesting that searches were 3.4% more accurate when targets were located out of the FOV than in. The interaction was not significant, F(1, 29) = 0.87, p = .36, $η_{p}^{2}$ = 0.03.

Figure 4.

Mean percent error for cue type (single, dual) and target object location (blue filled bars = in FOV; unfilled bars = out of FOV).

Next, the joint effect of cue type (single-dual) and reliability were examined for both response time and percent error using a 2 (cue type) × 2 (cue reliability) repeated-measures ANOVA. The no-cue condition was excluded.

Response Time

Figure 5 shows the mean response time for cue type and reliability.

Figure 5.

Mean response time for cue type (single, dual) and cue reliability (imperfect = gray, perfect = blue).

The ANOVA revealed that dual cues (M = 2.70 s) led to significantly faster searches than single cues (M = 3.62 s), F(1, 29) = 122.39, p < .001, $η_{p}^{2}$ = 0.81. The effect of reliability was significant, F(1, 29) = 44.43, p < .001, $η_{p}^{2}$ = 0.16, indicating that search time was 1.2 s longer with imperfect cues. The interaction was not statistically significant, F(1, 29) = 1.02, p = .32, $η_{p}^{2}$ = 0.03.

Percent Error

Figure 6 shows the mean percent error.

Figure 6.

Mean percent error for cue type (single, dual) and cue reliability (imperfect = gray, perfect = blue).

Like response time, dual cues significantly improved performance (reducing errors by 2.54%) compared to single cues, F(1, 29) = 11.59, p = .003, $η_{p}^{2}$ = 0.29. The results showed 6.4% greater errors when reliability was imperfect F(1, 29) = 41.44, p < .001, $η_{p}^{2}$ = 0.59. The interaction was not significant, F(1, 29) = 0.89, p = .35, $η_{p}^{2}$ = 0.03.

Automation Bias: Correct Versus Incorrect Automation Cueing

The imperfect cue condition alone was examined to assess the presence of the automation bias. Only the errors for the cued routine target were considered to assess the user’s reliance on the cues when they err by cueing nothing. The data were categorized based on the automation performance: when the automation was correct versus incorrect.

Percent Error

Figure 7 shows the mean percent error .

Figure 7.

Mean percent error for cue type (single, dual) and automation performance (correct = filled green bars, incorrect = unfilled red bars).

The ANOVA revealed a significant effect of automation performance, F(1, 29) = 111.69, p < .001, $η_{p}^{2}$ = 0.79, showing that percent error was much greater when automation was incorrect (M = 28.9%) than correct (M = 3.0%). The main effect of dual cueing was not significant, F(1, 29) = 0.58, p = 0.45, $η_{p}^{2}$ = 0.02. Critically, the interaction was significant, F(1, 29) = 8.53, p < .001, $η_{p}^{2}$ = 0.23. Pairwise comparisons (paired t-test) show that, when automation was incorrect, dual cues increased errors (M = 30.6%) more than single cues (M = 24.0%), t(29) = −2.03, p = .052, 95% CI [−0.13, 0.0006], d = 0.36. But when automation was correct, dual cues reduced errors (M = 1.4%) more than single cues (M = 5.4%), t(29) = 3.67, p < .001, 95% CI [0.018, 0.06], d = 0.61.

Discussion

The current work examined attention cue effectiveness during a wide FOV search task and how cue effectiveness is impacted by the number of cues and cue reliability.

Confirming H_1A and replicating prior work (Warden et al., 2023), cues improved search performance over an unaided (i.e., no-cue) search. Search tasks aided by an attention cue allowed for a 3.4 second faster and a 16.5% more accurate search compared to an unaided search. The significant improvement in search time and accuracy suggests that cues, even when imperfectly reliable, help the user allocate attentional resources more efficiently and reduce cognitive load when searching a complex scene, highlighting the effectiveness of cues in aiding search.

While cues improved overall search performance, participants exhibited a general automation bias when searching for targets with attention cues, supporting H_1B. Similar to prior work (Warden et al., 2023), participants made more errors when the automation was incorrect versus correct, and more critically, when the cues were incorrect they made more errors than the unaided search. This suggests a tendency to blindly follow the recommendation of the attention cue rather than confirm that the cued target was the correct target.

Overall, dual cues did improve performance more than single cues when targets were located both in and out of the FOV. However, disconfirming H₂, for both single and dual cues, response times were longer, and accuracy was higher when objects were outside the FOV. This finding suggests a speed-accuracy tradeoff where users prioritize accuracy over speed when objects are outside the FOV and more difficult to locate. While not statistically significant (p = 0.07), the pattern of results for response time suggests that dual cues reduce search time more than single cues when the target is located outside of the FOV. Unlike single cues, dual cues consist of a global and local cue, where the global cue can orient attention to targets outside of the FOV faster by indicating the general direction where the local cue, and hence the target, is positioned.

Confirming H_3A, dual cues improved search performance more than single cues when the cues were perfect, whereas performance degraded similarly for single and dual cues when the cues were imperfect. But, on trials when the automation erred in the imperfect condition, the automation bias was amplified by the more effective dual cues, supporting H_3B and replicating prior work showing that the cue that is best when perfect degrades performance most when it is wrong (Warden et al., 2023). Such an automation bias highlights the importance of mitigating overreliance when using attention guidance systems.

Conclusions and Limitations

These findings show the performance benefit of attention cues during a wide FOV visual search task, and highlight the potential for implementing dual cues in operational settings where the search field is large, and rapid and accurate search is critical, such as military or search and rescue applications. However, caution should be taken in these designs when the automated cues err in order to mitigate the automation bias, which is enhanced by the most effective cue. Future work should seek to examine larger search scenes, where targets are located beyond 128 degrees of visual angle (i.e., when using an AR-HMD). Additionally, future work should assess contexts where dual cues may be more beneficial and examine methods for mitigating the automation bias.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Binetti

Chen

Kruijff

Julier

Brumby

D. P.

(2021). Using visual and auditory cues to locate out-of-view objects in head-mounted augmented reality. Displays, 69, 102032. https://doi.org/10.1016/j.displa.2021.102032

Hein

Bullinger

A. C.

Bernhagen

(2020). Two is better than one: Improved attention guiding in AR by combining techniques. IEEE Computer Society, 40(5), 57–66.

LaBerge

Brown

(1989). Theory of attentional operations in shape identification. Psychological Review, 96(1), 101.

Mahoney

J. R.

Verghese

Dumas

Wang

Holtzer

(2012). The effect of multisensory cues on attention in aging. Brain Research, 1472, 63–73.

Posner

M. I.

(1980). Orienting of attention. Quarterly Journal of Experimental Psychology, 32(1), 3–25.

Raikwar

Warden

Misfud

Wickens

Batmaz

Kelley

Clegg

Ortega

(2024). Beyond the Wizard of Oz: Negative effects of imperfect machine learning to examine the impact of reliability of augmented reality cues on visual search performance. Transactions on Visualization and Computer Graphics, 30(5), 2662–2670. https://doi.org/10.1109/TVCG.2024.3372062

Santangelo

Spence

(2008). Capturing spatial attention with multisensory cues. Psychonomic Bulletin & Review, 15, 398–403.

Warden

A. C.

Wickens

C. D.

Clegg

B. A.

Rehberg

Ortega

F. R.

(2024). Information access effort: The role of head movements for information presented at increasing eccentricity on flat panel and head-mounted displays. Human Factors, 66(8), 2057–2081. https://doi.org/10.1177/00187208231204567

Warden

A. C.

Wickens

C. D.

Rehberg

Ortega

F. R.

Clegg

B. A.

(2023). Fast, accurate, but sometimes too-compelling support: The impact of imperfectly automated cues in an augmented reality head-mounted display on visual search performance. arXiv preprint arXiv:2303.14300.

10.

Wickens

C. D.

McCarley

J. S.

Gutzwiller

R. S.

(2022). Applied attention theory. CRC Press.

11.

Wiegmann

McCarley

J. S.

Kramer

A. F.

Wickens

C. D.

(2006). Age and automation interact to influence performance of a simulated luggage screening task. Aviation, Space, and Environmental Medicine, 77(8), 825–831.

12.

Wolfe

J. M.

(2021). Guided search 6.0: An updated model of visual search. Psychonomic Bulletin & Review, 28(4), 1060–1092.

13.

C. C.

Wolfe

J. M.

(2019). Eye movements in medical image perception: A selective review of past, present and future. Vision, 3(2), 32.

14.

Yeh

Wickens

C. D.

(2001). Display signaling in augmented reality: Effects of cue reliability and image realism on attention allocation and trust calibration. Human Factors, 43(3), 355–365. https://doi.org/10.1518/001872001775898269

Strategic Attention Guidance: The Impact of Dual and Single Cues in Wide Field of View Searches

Abstract

Keywords

Introduction

Method

Participants

Stimuli and Apparatus

Design

Task

Procedure

Results

Cue Effectiveness for Perfect Single Cue Conditions

Response Time

Percent Error

Effect of Dual Cueing

Response Time

Percent Error

Response Time

Percent Error

Automation Bias: Correct Versus Incorrect Automation Cueing

Percent Error

Discussion

Conclusions and Limitations

Footnotes

Declaration of Conflicting Interests

Funding

References