Crossmodal Links in Attention in the Driving Environment: The Roles of Cueing Modality,Signal Timing,and Workload

Abstract

Multimodal information presentation has been proposed as a means to support timesharing in complex data-rich environments. To ensure the effectiveness of this approach, it is necessary to consider performance effects of recently discovered crossmodal spatial and temporal links in attention, as well as their interaction with other performance-shaping factors. The main goals of this research were to confirm that performance effects of crossmodal links in spatial attention scale to complex environments and to examine how these effects vary as a function of cue modality, signal timing, and workload. In the present study, set in a driving simulation, spatially valid and invalid auditory and tactile cues preceded the presentation of visual targets at various stimulus-onset asynchronies and under different levels of workload induced by simulated wind gusts of varied intensity. The findings from this experiment confirm that visual target identification accuracies and response times are, overall, more accurate and faster when validly-cued. Significant interactions were found between cue validity, stimulus onset asynchrony (SOA), and cue modality, such that valid tactile cueing is most beneficial at shorter (100–200 ms) SOAs, while valid auditory cueing resulted in faster responses than invalid cueing at 500 ms SOAs, but slower responses at 1000 ms SOAs. Tactile error rates were significantly higher than auditory error rates at various interactions of modality and SOA. These findings were robust across all workload conditions. They highlight the need for context-sensitive information presentation and can inform the design of multimodal interfaces for a wide range of application domains.

Get full access to this article

View all access options for this article.

References

Ferris

Penfold

Hameed

Sarter

N.B.

(2006). The Implications of Crossmodal Links in Attention for the Design of Multimodal Interfaces: A Driving Simulation Study. Proceedings of the 50th Annual Meeting of the Human Factors and Ergonomics Society. San Francisco, CA. October.

Ferris

Sarter

N.B.

(2008). Crossmodal Links Between Vision, Audition, and Touch in Complex Environments. Human Factors, 50 (1), 17–26.

Hameed

Jayaraman

Ballard

Sarter

N.B.

(2007). Guiding Visual Attention by Exploiting Crossmodal Spatial Links: An Application in Air Traffic Control. Proceedings of the 51st Annual Meeting of the Human Factors and Ergonomics Society. Baltimore, MD. October.

Tan

H.Z.

Spence

(2006). The Differential Effect of Vibrotactile and Auditory Cues on Visual Spatial Attention. Ergonomics, 49, (7), 724–738.

Klein

R.M.

(2000). Inhibition of Return. Trends in Cognitive Sciences, 4 (4), 138–147.

Oviatt

(1999). Ten Myths of Multimodal Interactions. Communications of the ACM, 42 (11), 74–81.

Sarter

N.B.

(2002). Multimodal Information Presentation In Support of Human-Automation Communication and Coordination. In Salas

(Ed.), Advances in Human Performance and Cognitive Engineering Research. New York, NY: JAI Press. 13–36.

Spence

Driver

(1997). Cross-Modal Links in Attention Between Audition, Vision, and Touch: Implications for Interface Design. International Journal of Cognitive Ergonomics, 1 (4), 351–373.

Spence

Driver

(1998). Auditory and Audiovisual Inhibition of Return. Perception and Psychophysics, 60 (1), 125–139.

10.

Spence

McDonald

Driver

(2004). Exogenous spatial-cuing studies of human cross-modal attention and multisensory integration. In Spence

Driver

(Eds.), Crossmodal space and crossmodal attention. New York: Oxford University Press. 277–320.

11.

Spence

Pavani

Driver

(2000). Crossmodal Links Between Vision and Touch in Covert Endogenous Spatial Attention. Journal of Experimental Psychology: Human Perception and Performance, 26 (4), 1298–1319.