Exploiting the Auditory Modality in Decision Support

Abstract

The rate at which technology continues to develop and permeate our lives is such that it has become increasingly easier, and thus more likely, for information to be presented to us via different modalities simultaneously. But to what extent does this confluence of information affect our subsequent judgment and performance? Furthermore, what are the implications for system design when this information is critical to saving our lives and others? This study uses a visual ‘microworld’ simulation of a naval anti-air warfare to investigate whether the content and priority of audio messages that accompany changes in the visual modality assist or hinder performance in the task (identification of change and evaluation of threats). Results indicate that although helping critical change detection, a critical warning in the auditory modality is not as efficient as its visual counterpart. Moreover, audio messages tended to bias threat evaluation towards perceiving objects as more hostile than they were in reality. Such findings have clear implications in regard to the costs and benefits of further exploiting the auditory modality in dynamic visual environments.