Supporting Synchronous Distributed Communication and Coordination Through Multimodal Information Exchange

Abstract

Increasingly, operators in complex event-driven domains, such as the military, need to coordinate their goals and activities with numerous co-located and distributed human and machine agents. One promising way to support this requirement is the introduction of multimodal interfaces that afford functions such as increased bandwidth, complementarity, redundancy, and substitution. To inform the design of a robust multimodal system, the present study explored natural tendencies for, and the role of context in, modality usage in the context of simulated battlefield operations. Three groups of three ROTC cadets/officers each completed a set of 30-minute scenarios. The within-subject variables included participant location, the availability of radio communication, the amount of coordination required by the scenario, and the tempo of operations. Our findings show that participants were highly selective in their use of multiple modalities. Multimodal interaction was observed primarily in the context of spatial tasks and for the purpose of supporting grounding, complementarity, and disambiguation. Joint modality usage patterns evolved over time within groups and varied as a function of factors such as scenario and interface management demands. The findings from this study provide important guidance for the design of multimodal combined HCI and CSCW interfaces and show that multimodal information exchange is a highly effective means of coordination.