Defining Next-Generation Multi-Modal Communication in Human Robot Interaction

Abstract

With teleoperation being the contemporary standard for Human Robot Interaction (HRI), research into multi-modal communication (MMC) has focused on development of advanced Operator Control Units (OCU) supporting control of one or more robots. However, with advances being made to improve the perception, intelligence, and mobility of robots, a need exists to revolutionize the ways in which Soldiers interact with robotic team members. Within this future vision, mixed-initiative Soldier-Robot (SR) teams will work collaboratively sharing information back-and-forth in a fluid natural manner using combinations of communication methods. Therefore, new definitions are required to focus research efforts to support next-generation MMC. After a thorough survey of the literature and a scientific workshop on the topic, this paper aims to operationally define MMC, Explicit Communication, and Implicit Communication to encompass the shifting paradigm of HRI from a controller/controlled relationship to a cooperative team mate relationship. This paper presents the results from a survey of the literature and a scientific workshop that inform proposed definitions for multi-modal, explicit, and implicit communication. An illustrative scenario vignette provides context and specific examples of each communication type. Finally, future research efforts are summarized.

Get full access to this article

View all access options for this article.

References

Bauer

Wollherr

Buss

(2007). Human-robot collaboration: A survey. International Journal of Humanoid Robotics, 12(8), pp. 1-20.

Bischoff

Graefe

(2002). Dependable multimodal communication and interaction with robotic assistants. Proceedings of the 11th IEEE International Workshop on Robot and Human Interactive Communication, Berlin, 300-305.

Bloomfield

Badler

(2008) Virtual training via vibrotactile arrays. Presence J., 17(2), 103-120.

Elliott

L. R.

Coovert

M. D.

Prewett

Walvord

A. G.

Saboe

Johnson

(2009). A review and meta analysis of vibrotactile and visual information displays (ARL-TR-4955). Aberdeen Proving Ground, MD: U.S. Army Research Laboratory, 1-36.

Gunn

D. V.

Warm

J. S.

Nelson

W. T.

Bolia

R. S.

Schumsky

D. A.

Corcoran

K. J.

(2005). Target acquisition with UAVs: Vigilance displays and advanced cueing interfaces. Human Factors, 47(3), 488-497.

Guo

Sharlin

(2008). Exploring the use of tangible user interfaces for human-robot interaction: A comparative study. Proceedings of CHI’08, ACM, 1-10.

Haas

E. C.

(2007). Integrating auditory warnings with tactile cues in multimodal displays for challenging environments. Proceedings of the 13th International Conference on Auditory Displays, Montreal, Canada, 127-131.

Hancock

P.A.

(1996). On convergent technological evolution. Ergonomics in Design, 41, 22-9.

Hutchins

Cosenzo

McDermott

Feng

Barnes

Gacy

(2009). An investigation of the tactile communications channel for robotic control. Proceedings of Human Factors and Ergonomics Society Annual Meeting, 53(4), 182-186.

10.

Lin

Y. Y.

Becker

Makedon

(2010). Acoustical implicit communication in human-robot interaction. Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments (PETRA).

11.

Mehrabian

(1981). Silent messages: Implicit communication of emotions and attitudes. Belmont, CA: Wadsworth Pub. Co.

12.

Nigay

Coutaz

(1993). A Design Space for Multimodal Systems: Concurrent Processing and Data Fusion. In Proceedings of INTERCHI’93, Amsterdam, Netherlands, 172-178.

13.

Oviatt

(2002). Breaking the Robustness Barrier: Recent Progress on the Design of Robust Multimodal Systems. Advances in Computers. San Diego: CA. Elsevier Sciences.

14.

Oviatt

(2000). Taming Recognition Errors with a Multimodal Interface. Communications of the ACM, 43(9), 45-51.

15.

Oviatt

Coulston

Lunsford

(2004). When Do We Interact Multimodally? Cognitive Load and Multimodal Communication Patterns. Proceedings of the 6th International Conference on Multimodal Interfaces, New York, NY: ACM Press, 129-136.

16.

Parr

L. A.

(2004). Perceptual Biases for Multimodal Cues in Chimpanzee (Pan troglodytes) Affect Recognition. Animal Cognition, 7, 171-178.

17.

Pelechano

Allbeck

Badler

(2008). Virtual Crowds, Methods, Simulation and Control. Synthesis Lectures on Computer Graphics and Animation, Morgan & Claypool.

18.

Rani

Sarkar

Smith

C. A.

Adams

J. A.

(2003). Affective communication for implicit human-machine interaction. IEEE International Conference on Systems, Man and Cybernetics, 2003(5), 4896-4903.

19.

Redden

E. S.

Carstens

C. B.

Pettitt

(2010). Intuitive speech-based robotic control (ADA519652). Aberdeen Proving Ground, MD: U.S. Army Research Laboratory, 1-50.

20.

Redden

Elliott

(2010). Robotic Control Systems for Dismounted Soldiers. In Barnes

M. J.

Jentsch

F. G.

(eds), Human-Robot Interactions in Future Military Operations (pp. 335–352). Farnham: Ashgate Publishing.

21.

Richmond

V.P.

McCroskey

J.C.

Hickson

M.L.

(2008). Nonverbal Behavior in Interpersonal Relations. Boston: Allyn & Bacon.

22.

Wexelblat

(1995). An approach to natural gesture in virtual environments. ACM Transactions on Computer-Human Interaction, 2(3), 179-200.

23.

White

(2010). Suitable body locations and vibrotactile cueing types for dismounted soldiers. (ARL-TR-5186). Aberdeen Proving Ground, MD: U.S. Army Research Laboratory, 1-38.