The National Academies Board on Human-Systems Integration (BOHSI) Panel: Human-AI Teaming: Research Frontiers

Abstract

The National Academies recently issued a consensus study report summarizing the state of the art and research needs relating to Human-AI teaming, especially in the context of dynamic, high-risk environments such as multi-domain military operations. This consensus study was conducted by the National Academies Board on Human-Systems Integration (BOHSI). This panel, organized by BOHSI, brings together prominent researchers, including several members of the consensus committee, to discuss the state of the art and research frontiers for development of effective human-AI teams that can operate resiliently in complex, data intensive, and dynamically paced environments.

Get full access to this article

View all access options for this article.

References

Air Force Scientific Advisory Board. (2004). Human-System Integration in Air Force Weapon Development and Acquisition. Available: https://www.scientificadvisoryboard.af.mil/Studies/

Arrow

K. J.

(1974). The Limits of Organization. W. W. Norton & Company.

Beede

Baylor

Hersch

Iurchenko

Wilcox

Ruamviboonsuk

Vardoulakis

(2020). A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, April 25–30, Honolulu, HI, pp. 1–12.

Bhaskara

, et al. (2021). Effect of automation transparency in the management of multiple unmanned vehicles. Applied Ergonomics, 90, 103243.

Bisantz

Roth

E.M.

(2008). Analysis of Cognitive Work. Pp. 1–43 in Reviews of Human Factors and Ergonomics, vol. 3 ( Boehm-Davis

D.A.

, ed.). Santa Monica, CA: Human Factors and Ergonomics Society.

Boehm-Davis

D.A.

Durso

F.T.

Lee

J.D.

(2015). APA Handbook of Human Systems Integration. Washington DC: American Psychological Association.

Calhoun

Bartik

Ruff

, et al. Enabling human-autonomy teaming with multi-unmanned vehicle control interfaces. Hum.-Intell. Syst. Integr. 3, 155–174 (2021). https://doi.org/10.1007/s42454-020-00020-0

Chen

J. Y.

Lakhmani

S. G.

Stowers

Selkowitz

A. R.

Wright

J. L.

Barnes

(2018). Situation awareness-based agent transparency and human-autonomy teaming effectiveness. Theoretical issues in ergonomics science, 19(3), 259-282.

Chen

J. Y.

Barnes

M. J.

(2014). Human-agent teaming for multirobot control: A review of human factors issues. IEEE Transactions on Human-Machine Systems, 44, 13–29.

10.

Chien

Lewis

Sycara

Kumru

Liu

(2020). Influence of culture, transparency, trust, and degree of automation on automation use. IEEE Transactions on Human-Machine Systems, 50, 205–214.

11.

Chiou

E. K.

Lee

J. D.

(2021). Trusting automation: Designing for responsivity and resilience. Human Factors. https://doi.org/10/gjvcr2

12.

Cook

(2021). The future of artificial intelligence in ISR Operations. Air and Space Power Journal (Summer), 41–55. Available: https://www.airuniversity.af.edu/Portals/10/ASPJ/journals/Volume-35_Special_Issue/F-Cook.pdf

13.

Crandall

Klein

Hoffman

R.R.

(2006). Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. Cambridge, MA: MIT Press.

14.

Endsley

M.R.

(2019). Human factors and aviation safety: Testimony to the United States House of Representatives Hearing on Boeing 737-Max8 crashes. Available: https://transportation.house.gov/imo/media/doc/Endsley%20Testimony.pdf

15.

Endsley

M.R.

(2020). Human-Automation Interaction and the Challenge of Maintaining Situation Awareness in Future Autonomous Vehicles. In Mouloua

Hancock

(Eds.), Automation and Human Performance: Theory and Applications, 2nd ed. (pp. 151–168). Boca Raton, FL: CRC Press.

16.

Endsley

M.R.

(2017). From here to autonomy: Lessons learned from human-automation research. Human Factors, 59(1), 5–27.

17.

Endsley

M.R.

Jones

D.G.

(2012). Designing for Situation Awareness: An Approach to Human-Centered Design, 2nd ed. London: Taylor and Francis.

18.

Evenson

Muller

Roth

E.M.

(2008). Capturing the context of use to inform system design. Journal of Cognitive Engineering and Decision Making, 2(3), 181–203

19.

Flathmann

Schelble

B. G.

Zhang

McNeese

N. J.

(2021, July). Modeling and Guiding the Creation of Ethical Human-AI Teams. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 469-479).

20.

Hill

S. C.

(2021). Joint all-domain operations: The key to decision dominance and overmatch. Breaking Defense. Available: https://breakingdefense.com/2021/2005/joint-all-domain-operations-the-key-to-decision-dominance-and-overmatch/

21.

IJtsma

L.M.

Pritchett

A. R.

Feigh

K. M.

(2019). Computational methodology for the allocation of work and interaction in human-robot teams. Journal of Cognitive Engineering and Decision Making, 13(4), 221–241. doi:10.1177/1555343419869484

22.

Johnson

Bradshaw

J. M.

Feltovich

P. J.

(2017). Tomorrow’s human–machine design tools: From levels of automation to interdependencies. Journal of Cognitive Engineering and Decision Making, 12(1), 77–82.

23.

Kunze

Summerskill

S. J.

Marshall

, et al. (2019). Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces. Ergonomics, 62, 345-360.

24.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 31. https://doi.org/10.1518/hfes.46.1.50_30392

25.

Lyons

J. B.

(2013). Being Transparent About Transparency: A Model for Human-Robot Interaction. 2013 AAAI Spring Symposium Series, pp. 48–53.

26.

Lyons

J. B.

Wynne

K. T.

Mahoney

Roebke

M. A.

(2019). Trust and human-machine teaming: A qualitative study. In Artificial intelligence for the internet of everything (pp. 101-116). Academic Press.

27.

Lyons

J. B.

Sycara

Lewis

Capiola

(2021). Human-autonomy teaming: Definitions, debates, and directions. Frontiers in Psychology, 12, 1932.

28.

Matthews

Lin

Panganiban

Long

(2020). Individual differences in trust in autonomous robots: Implications for transparency. IEEE Transactions on Human-Machine Systems, 50, 234–244.

29.

McDermott

Dominguez

Kasdaglis

Ryan

Trahan

Nelson

2018. Human-Machine Teaming Systems Engineering Guide. Bedford, MA: MITRE.

30.

McNeese

N. J.

Demir

Chiou

E. K.

Cooke

N. J.

(2021). Trust and team performance in human–autonomy teaming. International Journal of Electronic Commerce, 25(1), 51-72.

31.

National Academies of Sciences, Engineering, and Medicine (2021). Human-AI Teaming: State of the Art and Research Needs. Washington, DC: The National Academies Press. https://doi.org/10.17226/26355

32.

National Research Council (NRC). 2007. Human-System Integration in the System Development Process: A New Look. National Academy Press.

33.

Naikar

Human–Automation Interaction in Self-Organizing Sociotechnical Systems. Journal of Cognitive Engineering and Decision Making. 2018;12(1):62-66. doi: 10.1177/1555343417731223

34.

O’Neill

McNeese

Barron

Schelble

(2020). Human–autonomy teaming: A review and analysis of the empirical literature. Human factors, 0018720820960865.

35.

Roth

E. M.

DePass

Harter

Scott

Wampler

2018. Beyond levels of automation: Developing more detailed guidance for human automation interaction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62(1), 150–154.

36.

Roth

E. M.

DePass

Scott

Truxler

Smith

Wampler

(2017). Designing Collaborative Planning Systems: Putting Joint Cognitive Systems Principles to Practice. In Smith

P. J.

Hoffman

R. R.

(Eds.) Cognitive Systems Engineering: The Future for a Changing World (Pp. 247–268). Boca Raton: Taylor and Francis, CRC Press.

37.

Roth

E. M.

Sushereba

Militello

L. G.

Diiulio

Ernst

(2019). Function allocation considerations in the era of human autonomy teaming. Journal of Cognitive Engineering and Decision Making, 13(4), 199–220.

38.

SAE International. (2019). SAE6906 Standard Practice for Human Systems Integration. Warrendale, PA: SEA International. Available: https://www.sae.org/standards/content/sae6906/

39.

Schelble

B. G.

Flathmann

McNeese

N. J.

Freeman

Mallick

(2022). Let’s Think Together! Assessing Shared Mental Models, Performance, and Trust in Human-Agent Teams. Proceedings of the ACM on Human-Computer Interaction, 6(GROUP), 1-29.

40.

Zhang

McNeese

N. J.

Freeman

Musick

(2021). “ An Ideal Human” Expectations of AI Teammates in Human-AI Teaming. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW3), 1-25.

41.

Trapsilawati

Wickens

Chen

C.-H.

(2017). Transparency and Conflict Resolution Automation Reliability in Air Traffic Control. 19th International Symposium on Aviation Psychology, Dayton, Ohio, pp. 419–424.

42.

USAF. (2015). Autonomous Horizons: The Way Forward. Washington, DC: Office of the U.S. Air Force Chief Scientist.

43.

van de Merwe

Mallam

Nazir

(2022). Agent Transparency, Situation Awareness, Mental Workload, and Operator Performance: A Systematic Literature Review. Human Factors.

44.

Wickens

C. D.

Helton

W.S.

Hollands

J. G.

Banbury

(2022). Engineering Psychology and Human Performance, (5th ed. New York: Routledge.