Communication Strategies in Human-Autonomy Teams During Technological Failures

Abstract

Objective

This study examines low-, medium-, and high-performing Human-Autonomy Teams’ (HATs’) communication strategies during various technological failures that impact routine communication strategies to adapt to the task environment.

Background

Teams must adapt their communication strategies during dynamic tasks, where more successful teams make more substantial adaptations. Adaptations in communication strategies may explain how successful HATs overcome technological failures. Further, technological failures of variable severity may alter communication strategies of HATs at different performance levels in their attempts to overcome each failure.

Method

HATs in a Remotely Piloted Aircraft System-Synthetic Task Environment (RPAS-STE), involving three team members, were tasked with photographing targets. Each triad had two randomly assigned participants in navigator and photographer roles, teaming with an experimenter who simulated an AI pilot in a Wizard of Oz paradigm. Teams encountered two different technological failures, automation and autonomy, where autonomy failures were more challenging to overcome.

Results

High-performing HATs calibrated their communication strategy to the complexity of the different failures better than medium- and low-performing teams. Further, HATs adjusted their communication strategies over time. Finally, only the most severe failures required teams to increase the efficiency of their communication.

Conclusion

HAT effectiveness under degraded conditions depends on the type of communication strategies enacted by the team. Previous findings from studies of all-human teams apply here; however, novel results suggest information requests are particularly important to HAT success during failures.

Application

Understanding the communication strategies of HATs under degraded conditions can inform training protocols to help HATs overcome failures.

Keywords

levels of automation perturbations remotely piloted aircraft systems team communication

Get full access to this article

View all access options for this article.

References

Anderson

J. R.

(2007). How can the human mind occur in the physical universe? Oxford University Press.

Ashby

W. R.

(1956). An introduction to cybernetics. John Wiley and Sons.

Ball

Myers

Heiberg

Cooke

Matessa

Freiman

Rodgers

(2010). The synthetic teammate project. Computational & Mathematical Organization Theory, 16(3), 271–299. https://doi.org/10.1007/s10588-010-9065-3

Cannon-Bowers

J. A.

Salas

(1990). Cognitive psychology and team training: Shared mental models in complex systems. In Paper presented at the 5th annual Conference of the Society for Industrial and Organizational Psychology. Miami, FL.

Chow

Christoffersen

Woods

D. D.

(2000). A model of communication in support of distributed anomaly response and replanning. In Proceedings of the Human Factors And Ergonomics Society - Annual Meeting (Vol. 44, No. 1, pp. 34–37). Sage CA, Sage Publications.

Cohen

(1977). Statistical power analysis for the behavioral sciences. Academic Press.

Cooke

Demir

McNeese

Gorman

Myers

(2020a). Human–autonomy teaming in remotely piloted aircraft systems operations under degraded conditions. Cognitive Engineering Research Institute.

Cooke

N. J.

Demir

Huang

(2020b). A framework for human-autonomy teaming research. In 22nd International Conference on Human-Computer Interaction, Copenhagen, Denmark.

Cooke

N. J.

Gorman

Pedersen

Winner

Duran

Taylor

Amazeen

P. A.

Andrews

D. H.

Rowe

(2007). Acquisition and retention of team coordination in command-and-control. Cognitive Engineering Research Inst Mesa Az.

10.

Cooke

N. J.

Gorman

J. C.

Myers

C. W.

Duran

J. L.

(2013). Interactive team cognition. Cognitive Science, 37(2), 255–285. https://doi.org/10.1111/cogs.12009

11.

Cooke

N. J.

Shope

S. M.

(2004). Designing a synthetic task environment. In Schiflett

L. R. E.

Salas

Coovert

M. D.

(Eds.), Scaled worlds: Development, validation, and application (pp. 263–278). Ashgate Publishing.

12.

DeChurch

L. A.

Mesmer-Magnus

J. R.

(2010). Measuring shared team mental models: A meta-analysis. Group Dynamics: Theory, research, and practice, 14(1), 1–14. https://doi.org/10.1037/a0017455

13.

Demir

Cooke

N. J.

(2014). Human teaming changes driven by expectations of a synthetic teammate. In Proceedings of the Human Factors and Ergonomics Society - Annual Meeting (Vol. 58, No. 1, pp. 16–20). Sage CA, Sage Publications.

14.

Demir

McNeese

N. J.

Cooke

N. J.

(2017). Team situation awareness within the context of human-autonomy teaming. Cognitive Systems Research, 46, 3–12. https://doi.org/10.1016/j.cogsys.2016.11.003

15.

Demir

McNeese

N. J.

Cooke

N. J.

(2016). Team communication behaviors of the human-automation teaming. In 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA) (pp. 28–34). IEEE.

16.

Enders

C. K.

(n.d.). Three-factor mixed ANOVA (one between- and two within-subjects factors). Applied Missing Data .com: Companion website for Applied Missing Data Analysis. https://www.appliedmissingdata.com/three-factor-mixed-anova.html

17.

Entin

E. E.

Serfaty

(1999). Adaptive team coordination. Human Factors: The Journal of the Human Factors and Ergonomics Society, 41(2), 312–325. https://doi.org/10.1518/001872099779591196

18.

Gorman

J. C.

Demir

Cooke

N. J.

Grimm

D. A.

(2019). Evaluating socio-technical dynamics in a simulated remotely-piloted aircraft system: A layered dynamics approach. Ergonomics, 62(5), 629–643. https://doi.org/10.1080/00140139.2018.1557750

19.

Gorman

J. C.

Cooke

N. J.

Winner

J. L.

(2006). Measuring team situation awareness in decentralized command and control environ- ments. Ergonomics, 49(12-13), 1312–1325. https://doi.org/10.1080/00140130600612788

20.

Grimm

Demir

Gorman

J. C.

Cooke

N. J.

(2018). Systems level evaluation of resilience in human-autonomy teaming under degraded conditions. In 2018 resilience week (RWS) (pp. 124–130). IEEE.

21.

Hastie

Tibshirani

Friedman

J. H.

Friedman

J. H.

(2009). The elements of statistical learning: Data mining, inference, and prediction (vol. 2, pp. 1–758). Springer.

22.

Kaber

D. B.

(2018). A conceptual framework of autonomous and automated agents. Theoretical Issues in Ergonomics Science, 19(4), 406–430. https://doi.org/10.1080/1463922x.2017.1363314

23.

Kelley

J. F.

(1983). An empirical methodology for writing user-friendly natural language computer applications. Proceedings of ACM SIG-CHI '83Human Factors in Computing Systems (pp. 193–196). ACM.

24.

MacMillan

Entin

E. E.

Serfaty

(2004). Communication overhead: The hidden cost of team cognition. In Salas

Fiore

S. M.

(Eds.), Team cognition: Understanding the factors that drive process and performance (pp. 61–82). American Psychological Association. https://doi.org/10.1037/10690-004

25.

McNeese

Demir

Chiou

Cooke

Yanikian

(2019). Understanding the role of trust in human-autonomy teaming. Proceedings of the 52nd Hawaii International Conference on System Sciences, 6, 254–263. https://doi.org/10.24251/hicss.2019.032

26.

McNeese

N. J.

Demir

Cooke

N. J.

Myers

(2018). Teaming with a synthetic teammate: Insights into human-autonomy teaming. Human Factors, 60(2), 262–273. https://doi.org/10.1177/0018720817743223

27.

National Academies of Sciences . (2021). Engineering and medicine. In Human-AI teaming: State of the art and research needs. The National Academies Press. https://doi.org/10.17226/26355

28.

O’Neill

McNeese

Barron

Schelble

(2022). Human–autonomy teaming: A review and analysis of the empirical literature. Human Factors, 64(5), 904–938. https://doi.org/10.1177/0018720820960865

29.

Orasanu

J. M.

(1990). Shared mental models and crew decision making (CSL Report No. 46). Princeton University, Cognitive Science Laboratory.

30.

R Core Team . (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

31.

Riek

L. D.

(2012). Wizard of Oz studies in Hri: A systematic review and new reporting guidelines. Journal of Human-Robot Interaction, 1(1), 119–136. https://doi.org/10.5898/jhri.1.1.riek

32.

Sarstedt

Mooi

(2014). Cluster analysis. In A concise guide to market research (pp. 273–324). Springer. https://doi.org/10.1007/978-3-642-53965-7_9

33.

Shively

R. J.

Lachter

Koteskey

Brandt

S. L.

(2018). Crew resource management for automated teammates (CRM-A). In International conference on engineering Psychology and cognitive ergonomics (pp. 215–229). Springer.

34.

Song

Gyory

J. T.

Zhang

Soria Zurita

N. F.

Stump

Martin

Miller

Balon

Yukish

McComb

Cagan

(2022). Decoding the agility of artificial intelligence-assisted human design teams. Design Studies, 79(1), 101094. https://doi.org/10.1016/j.destud.2022.101094

35.

Tenhundfeld

ChatGPT . (2023). Two birds with one stone: Writing a paper entitled “Chat GPT as a tool for studying human-AI interaction in the wild” with ChatGPT. https://doi.org/10.13140/RG.2.2.25319.73123

36.

UCLA: Statistical Consulting Group . (n.d.). SPSS Mixed Command. https://stats.idre.ucla.edu/spss/seminars/spss-mixed-command/

37.

Wardat

Tashtoush

M. A.

AlAli

Jarrah

A. M.

(2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272