Limited-Teamwork Autonomy in Training: Transfer Effects on Performance and Coordination in Subsequent All-Human Teams

Abstract

This study examines how training with autonomous teammates who lack teamwork skills influences performance and coordination when transitioning to all-human teams. Participants operated in a simulated Remotely Piloted Aircraft System environment, first training with either a human or AI pilot before completing a final mission with a human-only team. Results revealed no significant differences in final outcome performance between groups. However, AI-trained teams exhibited reduced process-based efficiency, weaker anticipation, and lower influence dynamics—indicating limited transfer of collaborative skills. These findings highlight the importance of embedding adaptive communication, trust calibration, and interactive team cognition into AI training agents. Practical implications span high-reliability domains such as healthcare, aviation, and defense, where structured AI training must be paired with strategies that foster transferable teamwork capabilities. Future research should explore longitudinal effects and domain-specific applications of HAT-to-HHT training transfer.

Keywords

human autonomous teaming AI performance coordination team training

Get full access to this article

View all access options for this article.

References

Cooke

N. J.

Shope

S. M.

(2005). Synthetic task environments for teams: CERTT’s UAV-STE. In Stanton

Hedge

Brookhuis

Salas

Hendrick Editor

(Eds.), Handbook of human factors and ergonomics methods (pp. 41–46). CRC Press.

Cooke

N. J.

Gorman

J. C.

Myers

C. W.

Duran

J. L.

(2013). Interactive team cognition. Cognitive Science, 37(2), 255–285.

Cooke

Demir

Huang

(2020). A framework for human-autonomy team research. In Engineering psychology and cognitive ergonomics. Cognition and design: 17th international conference, EPCE 2020, Held as part of the 22nd HCI international conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020 (pp. 134–146). Springer International Publishing.

Dennis

A. R.

Lakhiwal

Sachdeva

(2023). AI agents as team members: Effects on satisfaction, conflict, trustworthiness, and willingness to work with. Journal of Management Information Systems, 40(2), 307–337.

Demir

McNeese

N. J.

Gorman

J. C.

Cooke

N. J.

Myers

C. W.

Grimm

D. A.

(2021). Exploration of teammate trust and interaction dynamics in human-autonomy teaming. IEEE Transactions on Human-Machine Systems, 51(6), 696–705.

Entin

E. E.

Serfaty

(1999). Adaptive team coordination. Human Factors, 41(2), 312–325.

Flathmann

Duan

Mcneese

N. J.

Hauptman

Zhang

(2024). Empirically understanding the potential impacts and process of social influence in human-AI teams. In Proceedings of the ACM on human-computer interaction, 8(CSCW1) (pp. 1–32). Association for Computing Machinery.

Gorman

J. C.

Grimm

D. A.

Dunbar

T. A.

(2018). Defining and measuring team effectiveness in dynamic environments and implications for team ITS. In Johnston

Sottilare

Sinatra

A. M.

Burke

C. S.

(Eds.), Building intelligent tutoring systems for teams. Emerald Publishing Limited.

Gutzwiller

R. S.

Yousefi

Larson-Calcao

Lee

J. R.

Verma

Tenhundfeld

N. L.

Maknojia

. (2019). Systematic review of the use and modification of the “Trust in Automated System Scale”. In Proceeding of the human factors and ergonomics society annual meeting (p. 10711813251357911). SAGE Publications.

10.

Hart

S. G.

Staveland

L. E.

(1988). “Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock

P. A.

Meshkati

(Eds.), Advances in psychology (pp. 139–183). Elsevier. https://doi.org/10.1016/s0166-4115(08)62386-9

11.

Johnson

Bradshaw

J. M.

Feltovich

Jonker

van Riemsdijk

Sierhuis

(2011). The fundamental principle of coactive design: Interdependence must shape autonomy. In De Vos

(Ed.), Coin international workshops, LNAI (Vol. 6541, pp. 172–191). Springer.

12.

McNeese

N. J.

Demir

Cooke

N. J.

Myers

(2018). Teaming with a synthetic teammate: Insights into human-autonomy teaming. Human Factors, 60(2), 262–273.

13.

O’Neill

McNeese

Barron

Schelble

(2020). Human–autonomy teaming: A review and analysis of the empirical literature. Human Factors, 64(5), 904–938.

14.

Rieth

Hagemann

(2022). Automation as an equal team player for humans?–A view into the field and implications for research and practice. Applied Ergonomics, 98, Article 103552.

15.

Salas

Cooke

N. J.

Rosen

M. A.

(2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50(3), 540–547.

16.

Sanneman

Shah

J. A

. (2023). Validating metrics for reward alignment in human-autonomy teaming. Computer in Human Behaviour, 146, 107809.

17.

Swezey

R. W.

Salas

(Eds.). (1992). Teams: Their training and performance (pp. 3–29). Ablex Publishing.

18.

Tokadlı

Dorneich

M. C.

(2019). Interaction paradigms: From human-human teaming to human-autonomy teaming. In 2019 IEEE/AIAA 38th digital avionics systems conference (DASC), San Diego, CA, USA. IEEE.

19.

Zhang

Chong

Kotovsky

Cagan

(2023). Trust in an AI versus a human teammate: The effects of teammate identity and performance on human-AI cooperation. Computers in Human Behavior, 139, Article 107536.