Responses to Alerts and Subjective Reports

Abstract

We conducted an experiment to obtain objective measures of responses to binary alerts and subjective reports of the effects of alerts. We focused on the effects of the diagnostic value of the alert. Alerts helped improve performance, and participants responded more strongly and faster to alerts with higher diagnostic value. However, they did not report a greater reduction in workload with these alerts. Results show partial dissociation between objective and subjective measures of responses to alerts; while compliance and reliance measures were consistent with reports of system reliance and rating, effects of mean screen time were inconsistent with reports of temporal demand, effort invested and task difficulty, and participants’ performance using the alert systems did not correspond with their satisfaction with their own performance. This dissociation should be considered when evaluating the effects of automation.

Get full access to this article

View all access options for this article.

References

Bettman

J. R.

Johnson

E. J.

Payne

J. W.

(1990). A componential analysis of cognitive effort in choice. Organizational Behavior and Human Decision Processes, 45, 111–139.

Botzer

Meyer

Parmet

(2013). Mental effort in binary categorization aided by binary cues. Journal of Experimental Psychology: Applied, 19(1), 39-54.

Durso

F. T.

Alexander

(2010). Managing workload: Performance and situation awareness in aviation systems. In Salas

Maurin

(Eds.), Human Factors in Aviation (2nd ed., pp. 217–247). Burlington, MA: Academic Press.

Green

Swets

(1966). Signal Detection Theory and Psychophysics. New York: Wiley.

Guttman

Smith

D. E.

(1969). Investigation of rules for dealing with outliers in small samples from the normal distribution: I: Estimation of the mean. Technometrics, 11(3), 527-550.

Hart

S. G.

Staveland

L. E.

(1988). Development of the NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock

P. A.

Meshkati

(Eds.), Human Mental Workload (pp. 139–183). Amsterdam, The Netherlands: North Holland.

Meyer

(2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46(2), 196-204.

Navon

Gopher

(1979). On the economy of the human processing system. Psychological Review, 86, 214-255.

Parasuraman

Sheridan

T. B.

Wickens

C. D.

(2000, May). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 30(3), 286-297.

10.

Tse

D. K.

Wilton

P. C.

(1988). Models of consumer satisfaction formation: An extension. Journal of Marketing Research, 25(2), 204-212.

11.

Vashitz

Meyer

Parmet

Peleg

Goldfarb

Porath

Gilutz

(2009). Defining and measuring physicians’ responses to clinical reminders. Journal of Biomedical Informatics, 42 (2), 317-326.

12.

Wickens

C. D.

Dixon

S. R.

(2007). The benefits of imperfect diagnostic automation: a synthesis of the literature. Theoretical Issues in Ergonomics Science, 8(3), 201-212.

13.

Wiczorek

Meyer

Guenzler

(2012). On the relation between reliance and compliance in an aided visual scanning task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 56(1), 253-257.

14.

Wilson

T. D.

Nisbett

R. E.

(1978). The accuracy of verbal reports about the effects of stimuli on evaluations and behavior. Social Psychology, 41(2), 118-131.