Decision Deferral in a Human-AI Joint Face-Matching Task: Effects on Human Performance and Trust

Abstract

This study investigates how human performance and trust are affected by the decision deferral rates of an AI-enabled decision support system in a high criticality domain such as security screening, where ethical and legal considerations prevent full automation. In such domains, deferring cases to a human agent becomes an essential process component. However, the systemic consequences of the rate of deferrals on human performance are unknown. In this study, a face-matching task with an automated face verification system was designed to investigate the effects of varying deferral rates. Results show that higher deferral rates are associated with higher sensitivity and higher workload, but lower throughput and lower trust in the AI. We conclude that deferral rates can affect performance and trust perceptions. The tradeoffs between deferral rate, sensitivity, throughput, and trust need to be considered in designing effective human-AI work systems.