A Qualitative Analysis of Trust Dynamics in Human-Agent Teams (HATs)

Abstract

This study investigated how and why trust changes over time between differing levels of autonomy (LOA) for an intelligence, surveillance, and reconnaissance (ISR) task involving four unmanned aerial vehicles (UAV). Four LOAs were implemented in the study ranging from low to high (manual, advice, consent, veto). The analysis examined if trust increased, decreased, or stayed the same during transitions between two different LOAs. Through a thematic analysis, themes influencing the participant’s trust in agents were identified. Current findings suggest trust increases most when a participant moves between moderate LOAs. Specifically, moving from the condition in which the role of the agent is to provide advice, to the agent making decisions on their own with the consent of the human. Trust was also shown to increase when the human moved from the highest LOA (human merely vetoes) to the manual condition. The most prevalent themes influencing trust included perceived agent errors and false alarms.

Get full access to this article

View all access options for this article.

References

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.

Chen

J. Y. C.

Barnes

M. J.

(2012). Supervisory control of multiple robots: Effects of imperfect automation and individual differences. Human Factors, 54(2), 157174.https://doi.org/10.1177/0018720811435843

de Visser

E. J.

Pak

Shaw

T. H.

(2018). From ‘automation’ to ‘autonomy’: The importance of trust repair in human-machine interaction. Ergonomics, 61(10), 1409-1427.

Guo

Yang

X. J.

(2020). Modeling and predicting trust dynamics in human–robot teaming: A Bayesian inference approach. International Journal of Social Robotics, 1-11.

Huang

Cummings

M. L.

Ono

(2020). A mixed analysis of influencing factors for trust in a risk-aware autonomy. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 64, No. 1, pp. 102-106). Sage CA: Los Angeles, CA: SAGE Publications.

Joe

J. C.

O’Hara

Medema

H. D.

Oxstrand

J. H.

(2014). Identifying requirements for effective human-automation teamwork (INL/CON-14-31340). Idaho National Lab. (INL), Idaho Falls, ID (United States). https://www.osti.gov/biblio/1149012

Johnson

Bradshaw

J. M.

Feltovich

P. J.

Jonker

van Riemsdijk

Sierhuis

(2012). Autonomy and interdependence in human-agent-robot teams. IEEE Intelligent Systems, 27(2), 43–51. https://doi.org/10.1109/MIS.2012.1

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46. 50-80.

Nam

Lewis

Sycara

(2018). Trust of humans in supervisory control of swarm robots with varied levels of autonomy. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 825-830). IEEE.

10.

Porat

Gilad

Rottem-Hovey

Silbiger

(2016). Supervising and controlling unmanned systems: A multi-phase study with subject matter experts. Frontiers in Psychology. 7(568). doi:10.3389/fpsyg.2016.00568

11.

Rebensky

Carmody

Ficke

Carroll

(2022). Teammates instead of tools: The impacts of level of autonomy on mission performance and human-agent teaming dynamics in multi-agent distributed teams. Frontiers.

12.

Ruff

H. A.

Narayanan

Draper

M. H.

(2002). Human interaction with levels of automation and decision-aid fidelity in the supervisory control of multiple simulated unmanned air vehicles. Presence: Teleoperators & Virtual Environments, 11(4), 335-351.

13.

Schelble

B. G.

Flathmann

McNeese

N. J.

Freeman

Mallick

(2022). Let’s think together! Assessing shared mental models, performance, and trust in human-agent teams. Proceedings of the ACM on Human-Computer Interaction, 6(GROUP), 1-29.

14.

U.S Air Force. (2016). Small, unmanned aircraft systems (SUAS) flight plan: 2016-2036.

15.

Wohleber

R. W.

Matthews

Lin

Szalma

J. L.

Calhoun

G. L.

Funke

G. J.

Chiu

C.-Y. P.

Ruff

H. A.

(2019).Vigilance and automation dependence in operation of multiple unmanned aerial systems (UAS): A simulation study. Human Factors, 61(3), 488–505.