Sage Journals: Discover world-class research

Abstract

Human-AI teaming refers to systems in which humans and artificial intelligence (AI) agents collaborate to provide significant mission performance improvements over that which humans or AI can achieve alone. The goal is faster and more accurate decision-making by integrating the rapid data ingest, learning, and analyses capabilities of AI with the creative problem solving and abstraction capabilities of humans. The purpose of this panel is to discuss research directions in Trust Engineering for building appropriate bi-directional trust between humans and AI. Discussions focus on the challenges in systems that are increasingly complex and work within imperfect information environments. Panelists provide their perspectives on addressing these challenges through concepts such as dynamic relationship management, adaptive systems, co-discovery learning, and algorithmic transparency. Mission scenarios in command and control (C2), piloting, cybersecurity, and criminal intelligence analysis demonstrate the importance of bi-directional trust in human-AI teams.

Get full access to this article

View all access options for this article.

References

Bhattacharya

Devinney

T. M.

Pillutla

M. M.

(1998). A formal model of trust based on outcomes. Academy of Management Review, 23(3), 459-472.

Bruni

(2018). Why Automated Agents Should Be More Like Alfred Pennyworth. In Proceedings of the 2018 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, FL.

Chen

Mac Aodha

Perona

Yue

(2018). Near-Optimal Machine Teaching via Explanatory Teaching Sets. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in PMLR 84:1970-1978.

Chen

J. Y.

Barnes

M. J.

Wright

J. L.

Stowers

Lakhmani

S. G.

(2017, May). Situation awareness-based agent transparency for human-autonomy teaming effectiveness. In Micro-and Nanotechnology Sensors, Systems, and Applications IX (Vol. 10194, p. 101941V). International Society for Optics and Photonics.

Cooke

N. J.

Gorman

J. C.

Myers

C. W.

Duran

J. L.

(2013). Interactive team cognition. Cognitive Science, 37(2), 255-285.

Cummings

M.L.

Bruni

(2010). Human-Automation Collaboration in Complex Multivariate Resource Allocation Decision Support Systems. International Journal of Intelligent Decision Technologies, 4(2), 101-114.

Dai

Lin

Bertino

Kantarcioglu

(2008, August). An approach to evaluate data trustworthiness based on data provenance. In Workshop on Secure Data Management (pp. 82-98). Springer, Berlin, Heidelberg.

Feigh

K. M.

Dorneich

M. C.

Hayes

C. C.

(2012). Toward a characterization of adaptive systems: A framework for researchers and system designers. Human Factors, 54(6), 1008-1024.

Gilpin

L. H.

Bau

Yuan

B. Z.

Bajwa

Specter

Kagal

(2018, October). Explaining Explanations: An Overview of Interpretability of Machine Learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) (pp. 80-89). IEEE.

10.

Gunning

(2017). Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency, DARPA/I20.

11.

Hayes

C. C.

Miller

C. A.

(2010). Human-computer etiquette: Cultural expectations and the design implications they place on computers and technology. CRC Press.

12.

Zhang

Ren

Sun

(2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1026-1034).

13.

Hoc

J. M.

(2001). Towards a cognitive approach to human–machine cooperation in dynamic situations. International Journal of Human-Computer Studies, 54(4), 509-540.

14.

Knepper

R. A.

Mavrogiannis

C. I.

Proft

Liang

(2017, March). Implicit communication in a joint action. In 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 283-292). IEEE.

15.

Lim

Ramasamy

Gardi

Kistan

Sabatini

(2018). Cognitive Human-Machine Interfaces and Interactions for Unmanned Aircraft. Journal of Intelligent & Robotic Systems, 91(3-4), 755-774.

16.

Lyons

J. B.

Koltai

K. S.

N. T.

Johnson

W. B.

Smith

D. E.

Shively

R. J.

(2016). Engineering trust in complex automated systems. Ergonomics in Design, 24(1), 13-17.

17.

Marcus

(2018). Deep Learning: A Critical Appraisal. arXiv:1801.00631

18.

MOD, Development, Concepts and Doctrine Centre. 2018. Human-Machine Teaming. Joint Concept Note 1/18. Ministry of Defence, Arncott, UK.

19.

Onnasch

Wickens

C. D.

Manzey

(2014). Human performance consequences of stages and levels of automation: An integrated meta-analysis. Human Factors, 56(3), 476-488.

20.

Parasuraman

Riley

(1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230-253.

21.

Schaefer

K. E.

Chen

J. Y.

Szalma

J. L.

Hancock

P. A.

(2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors, 58(3), 377-400.

22.

Seshia

S. A.

Sadigh

Sastry

S. S.

(2016). Towards verified artificial intelligence. arXiv:1606.08514.

23.

Stacy

Freeman

(2016). Training objective packages: enhancing the effectiveness of experiential training. Theoretical Issues in Ergonomics Science, 17(2), 149-168.

24.

Swain

Mills

(2003). Implicit communication in novice and expert teams (No. DSTO-TN-0474). Defence Science and Technology Organisation, Salisbury (Australia) Systems Sciences Lab.

Trust Engineering for Human-AI Teams

Abstract

Get full access to this article

References