Technology users are increasingly given discretion around when to engage automated systems, however, these decisions are often suboptimal, inhibiting performance. Prior work has suggested that this may stem from improper estimates of task difficulty. This study investigated where perceived task difficulty diverged from objective task difficulty (accuracy) and its impact on automation use. In a visual search task, participants identified target objects in cluttered images and could opt to use an automated aid on certain trials. Results indicated that participants were sensitive to differences in difficulty, but they used the automation more often when they rated the task as more difficult. With automation present, performance was equivalent between the easy and difficult conditions, but participants rated the difficult condition as harder. This may suggest a disconnect between perceived difficulty and objective performance measures. These results highlight the importance of considering both subjective and objective difficulty when designing automated systems.
AdlerR. F.Benbunan-FichR. (2015). The effects of task difficulty and multitasking on performance. Interacting with Computers, 27(4), 430–439.
2.
BartlettM. L.McCarleyJ. S. (2017). Benchmarking aided decision making in a signal detection task. Human Factors, 59, 881–900.
3.
BirosD. P.DalyM.GunschG. (2004). The influence of task load and automation trust on deception detection. Group Decision and Negotiation, 13(2), 173–189.
4.
BoskemperM. M.BartlettM. L.McCarleyJ. S. (2021). Measuring the efficiency of automation-aided performance in a simulated baggage screening task. Human Factors: The Journal of the Human Factors and Ergonomics Society, 64(6), 945–961. https://doi.org/10.1177/0018720820983632
5.
CarragherD. J.SturmanD.HancockP. J. (2024). Trust in automation and the accuracy of human–algorithm teams performing one-to-one face matching tasks. Cognitive Research: Principles and Implications, 9(1), 41.
6.
de VriesP.MiddenC.BouwhuisD. (2003). The effects of errors on system trust, self-confidence, and the allocation of control in route planning. International Journal of Human-Computer Studies, 58, 719–735.
7.
FittsP. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), 381.
8.
HeG.KuiperL.GadirajuU. (2023). Knowing about knowing: An illusion of human competence can hinder appropriate reliance on AI systems. In Proceedings of the 2023 CHI conference on human factors in computing systems (pp. 1–18). Association for Computing Machinery.
9.
LeeJ. D.MorayN. (1992). Operators’ reliance on automation during the control of a semi-automatic process. In [Proceedings] 1992 IEEE international conference on systems, man, and cybernetics (pp. 526–530). IEEE.
10.
LeeJ. D.MorayN. (1994). Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies, 40(1), 153–184.
11.
LiuY.WickensC. D. (1994). Mental workload and cognitive task automaticity: an evaluation of subjective and time estimation metrics. Ergonomics, 37(11), 1843–1854.
12.
McBrideS. E.RogersW. A.FiskA. D. (2011). Understanding the effect of workload on automation use for younger and older adults. Human Factors, 53(6), 672–686.
13.
ParasuramanR.RileyV. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253.
14.
PattonC. E.CleggB. A.DavisB. C.CaglarT.SiebertC.BlanchardN. (2025). The choice to use automation: Improvements from evidence accumulation. International Journal of Human–Computer Interaction, 41(1), 1–18.
15.
PattonC. E. (2023). Automation use: The roles of self-confidence, trust, and workload. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 67, No. 1, pp. 965–970). SAGE Publications.
16.
RiethM.OnnaschL.HagemannV. (2024). Adaptable automation for a more human-centered work design? Effects on human perception and behavior. International Journal of Human-Computer Studies, 186, 103246.
17.
RileyV. A. (1994). Human use of automation. University of Minnesota.
18.
SchwarkJ.DolgovI.GravesW.HorD. (2010). The influence of perceived task difficulty and importance on automation use. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 54, No. 19, pp. 1503–1507). SAGE Publications.
19.
WickensC. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2), 159–177.
20.
WickensC. D.DixonS. R. (2007). The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomics Science, 8(3), 201-212.
21.
WiczorekR.MeyerJ. (2019). Effects of trust, self-confidence, and feedback on the use of decision automation. Frontiers in Psychology, 10, 519.
22.
XuX.WickensC. D.RantanenE. M. (2007). Effects of conflict alerting system reliability and task difficulty on pilots’ conflict detection with cockpit display of traffic information. Ergonomics, 50(1), 112–130.
23.
YamaniY.McCarleyJ. S. (2018). Effects of task difficulty and display format on automation usage strategy: A workload capacity analysis. Human Factors, 60(4), 527–537.
24.
YasminR.HassanM. M.GrasselJ. T.BhogarajuH.EscobedoA. R.FuentesO. (2022). Improving crowdsourcing-based image classification through expanded input elicitation and machine learning. Frontiers in Artificial Intelligence, 5, 848056.
25.
YehY.WickensC. D. (1988). Dissociation of performance and subjective measures of workload. Human Factors, 30(1), 111–120.
