Objective: Two experiments were conducted in which participants navigated a simulated unmanned aerial vehicle (UAV) through a series of mission legs while searching for targets and monitoring system parameters. The goal of the study was to highlight the qualitatively different effects of automation false alarms and misses as they relate to operator compliance and reliance, respectively. Background: Background data suggest that automation false alarms cause reduced compliance, whereas misses cause reduced reliance. Method: In two studies, 32 and 24 participants, including some licensed pilots, performed in-lab UAV simulations that presented the visual world and collected dependent measures. Results: Results indicated that with the low-reliability aids, false alarms correlated with poorer performance in the system failure task, whereas misses correlated with poorer performance in the concurrent tasks. Conclusion: Compliance and reliance do appear to be affected by false alarms and misses, respectively, and are relatively independent of each other. Application: Practical implications are that automated aids must be fairly reliable to provide global benefits and that false alarms and misses have qualitatively different effects on performance.
Get full access to this article
View all access options for this article.
References
1.
Bainbridge, L. (1983). Ironies of automation. Automatica, 19, 775–779.
2.
Bliss, J. (2003). An investigation of alarm related accidents and incidents in aviation. International Journal of Aviation Psychology, 13, 249–268.
3.
Breznitz, S. (1983). Cry-wolf: The psychology of false alarms.Hillsdale, NJ: Erlbaum.
4.
Dixon, S. R., Wickens, C. D., & Chang, D. (2005). Mission control of multiple unmanned aerial vehicles: A workload analysis. Human Factors, 47, 479–487.
5.
Galster, S. M., Bolia, R. S., Roe, M. M., & Parasuraman, R. (2001). Effects of automated cueing on decision implementation in a visual search task. In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting (pp. 321–325). Santa Monica, CA: Human Factors and Ergonomics Society.
6.
Green, D. M., & Swets, J. A. (1988). Signal detection theory and psychophysics.New York: Wiley.
7.
Keppel, G. (1982). Design and analysis: A researcher's handbook (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.
8.
Maltz, M., & Meyer, J. (2003). Use of warnings in an attentionally demanding detection task. Human Factors, 43, 217–226.
9.
Maltz, M., & Shinar, D. (2003). New alternative methods in analyzing human behavior in cued target acquisition. Human Factors, 45, 281–295.
10.
Masalonis, A. J., & Parasuraman, R. (1999). Trust as a construct for evaluation of automation aids. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 184–188). Santa Monica, CA: Human Factors and Ergonomics Society.
11.
Metzger, U., & Parasuraman, R. (2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47, 35–49.
12.
Meyer, J. (2001). Effects of warning validity and proximity on responses to warnings. Human Factors, 43, 563–572.
13.
Meyer, J. (2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46, 196–204.
14.
Molloy, R., & Parasuraman, R. (1996). Monitoring an automated system for a single failure: Vigilance and task complexity effects. Human Factors, 38, 211–322.
15.
Parasuraman, R., Molloy, R., & Singh, I.L. (1993). Performance consequences of automation-induced “complacency”. International Journal of Aviation Psychology, 3, 1–23.
16.
Parasuraman, R., & Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39, 230–253.
17.
St. John, M., & Manes, D. I. (2002). Making unreliable automation useful. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 332–336). Santa Monica, CA: Human Factors and Ergonomics Society.
18.
Sorkin, R. D. (1989). Why are people turning off alarms?Human Factors Society Bulletin, 32 (4), 3–4.
19.
Sorkin, R. D., & Woods, D. D. (1985). Systems with human monitors: A signal detection analysis. Human-Computer Interaction, 1, 49–75.
20.
Swets, J. A., & Pickett, R. M. (1982). The evaluation of diagnostic systems.New York: Academic Press.
21.
Wickens, C. D., & Dixon, S. R. (2005). Is there a magic number 7 (to the minus 1)? The benefits of imperfect diagnostic automation: A synthesis of the literature (Tech. Rep. AHFD-05-01/MAAD-05-1). Savoy, IL: University of Illinois, Aviation Human Factors Division.
22.
Wickens, T. D. (2002). Elementary signal detection theory.New York: Oxford University Press.
23.
Yaacov, A. B., Maltz, M., & Shinar, D. (2003). Effects of an in-vehicle collision avoidance warning system on short- and long-term driving performance. Human Factors, 44, 335–342.
24.
Yeh, M., Merlo, J., Wickens, C.D., & Brandenburg, D. L. (2003). Head up versus head down: The costs of imprecision, unreliability, and visual clutter on cue effectiveness for display signaling. Human Factors, 45, 390–407.
