Trust in Automation: The Effects of System Certainty on Decision-Making

Abstract

In the field of Human Factors, the concept of trust in automation can help to explain how and why users interact with particular systems. One way to examine trust is through task performance and/or behavioral observations. Previous work has identified several system-related moderators of trust in automation, such as reliability and complexity. However, the effects of system certainty, i.e., the knowledge that a machine has regarding its own decision-making abilities, on trust remains unclear. The goal of this study was to examine the extent to which system certainty affects perceived trust. Participants performed a partially simulated flight task and decided what action to take in response to targets in the environment detected by the aircraft’s automation. The automation’s certainty levels in recognizing targets were 30%, 50%, and 80%. Overall, participants accepted the system’s recommendation regardless of the certainty level and trust in the system increased as the system’s certainty level increased. Results may help to inform the development of future autonomous systems.

Get full access to this article

View all access options for this article.

References

Beller

Heesen

Vollrath

(2013). Improving the driver– automation interaction: An approach using automation uncertainty. Human Factors, 55(6), 1130–1141.

Endsley

M. R.

(2017). From here to autonomy: lessons learned from human–automation research. Human Factors, 59(1), 5–27.

Endsley

M. R.

Kiris

E. O.

(1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37(2), 381–394.

French

Duenser

Heathcote

(2018). Trust in Automation. CSIRO Report EP184082.

Hancock

P. A.

Billings

D. R.

Schaefer

K. E.

Chen

J. Y. C.

De Visser

E. J.

Parasuraman

(2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors, 53(5), 517–527.

Helldin

Falkman

Riveiro

Davidsson

(2013). Presenting system uncertainty in automotive UIs for supporting trust calibration in autonomous driving. Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 210–217.

Hoff

K. A.

Bashir

(2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434.

Jian

J.-Y.

Bisantz

A. M.

Drury

C. G.

(2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71.

Lee

J. D.

Moray

(1994). Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies, 40(1), 153–184.

10.

Lee

J. D.

See

K. A.

(2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392

11.

Sarter

(2019). Eye tracking: a process-oriented method for inferring trust in automation as a function of priming and system reliability. IEEE Transactions on Human-Machine Systems, 49(6), 560–568.

12.

McBride

S. E.

Rogers

W. A.

Fisk

A. D.

(2011). Understanding the effect of workload on automation use for younger and older adults. Human Factors, 53(6), 672–686.

13.

McGuirl

J. M.

Sarter

N. B.

(2006). Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Human Factors, 48(4), 656–665.

14.

Muir

B. M.

Moray

(1996). Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39(3), 429–460.

15.

Seppelt

B. D.

Lee

J. D.

(2007). Making adaptive cruise control (ACC) limits visible. International Journal of Human-Computer Studies, 65(3), 192–205.

16.

Tomsett

Preece

Braines

Cerutti

Chakraborty

Srivastava

… Kaplan

(2020). Rapid trust calibration through interpretable and uncertainty-aware AI. Patterns, 1(4), 100049.