Effects of Automation Reliability on Human Monitoring Performance

Abstract

Human decisions to rely on automation of a simulated flight task were monitored in the present study. Participants performed the Multi Attribute Task (MAT) battery previously developed by Comstock and Arneguard (1990). Twenty-eight participants were tested for 7 different reliability levels. Decision accuracy, false alarm rates, and reaction time to simulated automation failures were collected for each participant over nine 10-minute sessions. The results indicated that the rate of automation failures detected varied inversely with automation reliability level. As automation level increased, detection rate decreased thereby indicating automation-induced monitoring inefficiency. However, the mean reaction time of correct detection rate decreased as automation reliability level increased. No significant effect of reliability was obtained for the false alarm rate data. The implications for training and systems design are also outlined.

Get full access to this article

View all access options for this article.

References

Chambers

Nagel

D. C.

(1985). Pilots of the future: Human or computer? Communications of the Association for Computing Machinery, 28, 1187–1199.

Comstock

J. R.

Arnegard

R. J.

(1990). Multi-attribute task battery. (Preliminary Technical Report). Hampton, VA: NASA Langley Research Center.

Davies

D. R.

Parasuraman

(1982). The psychology of vigilance. London: Wiley.

Lee

Moray

(1992). Trust, control strategies, and allocation of function in the human-machine systems, Ergonomics, 35, 1243–1270.

May

P. A.

Mollory

R. J.

Parasuraman

(1993). Effects of automation reliability and failure rate on monitoring performance in a multitask environment. Poster session at the 37^th Annual Meeting of the Human Factors and Ergonomics Society. Santa Monica, CA: Human Factors and Ergonomics Society.

Mosier

Skitka

L. J.

Korte

K. J.

(1994). Cognitive and social psychological issues in flight crew/automation interaction. In Mouloua

Parasuraman

(Eds.), Human performance in automated systems-. Current research and trends (pp. 191–197). Hillsdale, NJ: Erlbaum.

Mouloua

Parasuraman

(1994). Human performance in automated systems: Recent research and trends. Hilsdale, NJ: Erlbaum.

Mouloua

Koonce

, (1997). Human-interaction: Research and practice. Hillsdale, NJ: Erlbaum.

Muir

(1988). Trust between humans and machines, and the design of decision aides. In Holnagel

Mancini

Woods

(Eds.), Cognitive Engineering in complex dynamic worlds. London: Academic Press.

10.

Muir

(1989). Operator's trust in and use of automatic controllers in a supervisory process control task. Doctoral thesis. University of Toronto.

11.

Parasuraman

Bowers

J. C.

(1987). Attention and vigilance in human-computer interactions. In Gale

Christie

(Eds.), Psychophysiology and the Electronic Workplace. London: Wiley.

12.

Parasuraman

Molloy

Singh

(1993). Performance consequences of automation-induced “complacency”. The International Journal of Aviation Psychology, 3, 1–13.

13.

Parasuraman

Mouloua

Molloy

(1996). Effects of adaptive task allocation on monitoring of automated systems. Human Factors, 28, 665–679.

14.

Scerbo

Mouloua

(1999). Automation technology and human performance: Current research and trends. Hillsdale, NJ: Erlbaum.

15.

Singh

I. L.

Molloy

Parasuraman

(1993). Automation induced “complacency”: Development of a Complacency Potential Rating Scale. The International Journal of Aviation Psychology, 3, 111–121.

16.

Wiener

E. L.

(1988). Cockpit automation. In Wiener

E. L.

Nagel

D. C.

(Eds.), Human factors in aviation (pp. 433–461). San Diego: Academic.