Team Skill Metrics that Span Human-Human and Human-Agent Teams: An Initial Assessment

Abstract

Team skill metrics were operationalized by translating team constructs to metrics based on observable behaviors. As human coordination with autonomous agents turns to collaboration, humans may increasingly view agents as teammates. This transition will require agents to possess team skills and necessitate appropriate metrics for measuring team skills across human-agent and human-human teams. Thirty-eight teaming metrics were developed across five stages of teaming: preparation, execution, evaluation, adjustment, and team chemistry. Behaviors from 78 multiplayer gameplay videos were coded to establish which metrics could be measured via observable behaviors. An exploratory assessment demonstrated that the metrics captured teaming differences in team composition (human-human teams vs. human-agent teams) and three levels of team expertise. Results suggest that these team skill metrics could aid agent designers in anticipating the team dynamics of humans working with their agents.