We evaluated the sensitivity and calibration of users when estimating the reliability of an automated decision support system as exposure to the system and the saliency of errors varied.
Background
Although users are typically sensitive when estimating the reliability of an automated system, it is not known how exposure to a system or error saliency influences the estimate. Viewpoints of the “leaky integrator” and “intuitive statistician” suggest competing outcomes regarding estimates over time.
Methods
Participants identified matching images of MegaBlocks with assistance from an automated system. Participants experienced three levels of reliability and three levels of exposure (number of trials). They provided estimates of reliability, trust, and confidence in their estimates for each system. In Experiment 2, the saliency of false alarms and misses was manipulated. In Experiment 3, the saliency of these error types was matched.
Results
Reliability estimates were sensitive to true reliability but were lower with more salient errors. Exposure had no impact on the estimates of reliability or trust. Trust and confidence in the estimate of reliability increased with true reliability.
Conclusions
Estimates of reliability are generally sensitive to the true reliability of a system, although they can be hindered by memory limitations and/or saliency effects. Estimates of reliability are made very quickly and do not tend to change with exposure.
Applications
Training users on a new system may not require extensive exposure, but early interactions with a system should accurately represent the amount and types of errors that the system will make.
Azevedo-SaH.JayaramanS. K.EsterwoodC. T.YangX. J.RobertL. P.TilburyD. M. (2020). Comparing the effects of false alarms and misses on humans’ trust in (semi)autonomous vehicles. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 113–115). IEEE. https://doi.org/10.1145/3371382.3378371
2.
Barg-WalkowL. H.RogersW. A. (2016). The effect of incorrect reliability information on expectations, perceptions, and use of automation. Human Factors, 58(2), 242–260. https://doi.org/10.1177/0018720815610271
BeachL. R.ScoppT. S. (1968). Intuitive statistical inferences about variances. Organizational Behavior and Human Performance, 3(2), 109–123. https://doi.org/10.1016/0030-5073(68)90001-9
5.
BergerA.KieferM. (2021). Comparison of different response time outlier exclusion methods: A simulation study. Frontiers in Psychology, 12(1), Article 675558. https://doi.org/10.3389/fpsyg.2021.675558
6.
BlissJ. P.GilsonR. D.DeatonJ. E. (1995). Human probability matching behaviour in response to alarms of varying reliability. Ergonomics, 38(11), 2300–2312. (world). https://doi.org/10.1080/00140139508925269
7.
BowdenV. K.GriffithsN.StricklandL.LoftS. (2021). Detecting a single automation failure: The impact of expected (but not experienced) automation reliability. Human Factors, 65(4), 00187208211037188. https://doi.org/10.1177/00187208211037188
8.
BreznitzS. (2013). Cry wolf: The psychology of false alarms. Taylor & Francis.
9.
CarragherD. J.HancockP. J. B. (2023). Simulated automated facial recognition systems as decision-aids in forensic face matching tasks. Journal of Experimental Psychology: General, 152(5), 1286–1304. https://doi.org/10.1037/xge0001310
10.
ChanceyE. T.BlissJ. P.ProapsA. B.MadhavanP. (2015). The role of trust as a mediator between system characteristics and response behaviors. Human Factors, 57(6), 947–958. https://doi.org/10.1177/0018720815582261
11.
ChanceyE. T.BlissJ. P.YamaniY.HandleyH. A. H. (2017). Trust and the compliance–reliance paradigm: The effects of risk, error bias, and reliability on trust and dependence. Human Factors, 59(3), 333–345. https://doi.org/10.1177/0018720816682648
12.
ChavaillazA.WastellD.SauerJ. (2016). System reliability, performance and trust in adaptable automation. Applied Ergonomics, 52(1), 333–342. https://doi.org/10.1016/j.apergo.2015.07.012
13.
DavenportR. B.BustamanteE. A. (2010). Effects of false-alarm vs. miss-prone automation and likelihood alarm technology on trust, reliance, and compliance in a miss-prone task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 54(19), 1513–1517. https://doi.org/10.1177/154193121005401933
14.
DekkingM.KraaikampC.LopuhaäH. P.MeesterL. E. (Eds.), (2005). A modern introduction to probability and statistics: Understanding why and how. Springer. https://doi.org/10.1007/1-84628-168-7
15.
de VisserE. J.MonfortS. S.GoodyearK.LuL.O’HaraM.LeeM. R.ParasuramanR.KruegerF. (2017). A little anthropomorphism goes a long way: Effects of oxytocin on trust, compliance, and team performance with automated agents. Human Factors, 59(1), 116–133. https://doi.org/10.1177/0018720816687205
16.
DixonS. R.WickensC. D. (2006). Automation reliability in unmanned aerial vehicle control: A reliance-compliance model of automation dependence in high workload. Human Factors, 48(3), 474–486. https://doi.org/10.1518/001872006778606822
17.
DixonS. R.WickensC. D.McCarleyJ. S. (2007). On the independence of compliance and reliance: Are automation false alarms worse than misses?Human Factors, 49(4), 564–572. https://doi.org/10.1518/001872007X215656
DuN.HuangK. Y.YangX. J. (2020). Not all information is equal: Effects of disclosing different types of likelihood information on trust, compliance and reliance, and task performance in human-automation teaming. Human Factors, 62(6), 987–1001. https://doi.org/10.1177/0018720819862916
20.
DzindoletM. T.PierceL. G.BeckH. P.DaweL. A. (2002). The perceived utility of human and automated aids in a visual detection task. Human Factors, 44(1), 79–94. https://doi.org/10.1518/0018720024494856
21.
ErlickD. E. (1964). Absolute judgments of discrete quantities randomly distributed over time. Journal of Experimental Psychology, 67(5), 475–482. https://doi.org/10.1037/h0042698
22.
ForoughiC. K.SibleyC.BrownN. L.RoviraE.PakR.CoyneJ. T. (2019). Detecting automation failures in a simulated supervisory control environment. Ergonomics, 62(9), 1150–1161. https://doi.org/10.1080/00140139.2019.1629639
23.
FredricksonB. L.KahnemanD. (1993). Duration neglect in retrospective evaluations of affective episodes. Journal of Personality and Social Psychology, 65(1), 45–55. https://doi.org/10.1037/0022-3514.65.1.45
24.
HanssonP.JuslinP.WinmanA. (2008). The role of short-term memory capacity and task experience for overconfidence in judgment under uncertainty. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(5), 1027–1042. https://doi.org/10.1037/a0012638
25.
HoffK. A.BashirM. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570
26.
HollandC.PerryG.NeyedliH. F. (2024). Calibrating trust, reliance and dependence in variable-reliability automation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 68(1), 604–610. https://doi.org/10.1177/10711813241277531
27.
HutchinsonJ.StricklandL.FarrellS.LoftS. (2022). Human behavioral response to fluctuating automation reliability. Applied Ergonomics, 105(1), Article 103835. https://doi.org/10.1016/j.apergo.2022.103835
28.
HutchinsonJ.StricklandL.FarrellS.LoftS. (2023). The perception of automation reliability and acceptance of automated advice. Human Factors, 65(8), 1596–1612. https://doi.org/10.1177/00187208211062985
29.
KahnemanD.FredricksonB. L.SchreiberC. A.RedelmeierD. A. (1993). When more pain is preferred to less: Adding a better end. Psychological Science, 4(6), 401–405. https://doi.org/10.1111/j.1467-9280.1993.tb00589.x
30.
KalyugaS.ChandlerP.SwellerJ. (2004). When redundant on-screen text in multimedia technical instruction can interfere with learning. Human Factors: The Journal of the Human Factors and Ergonomics Society, 46(3), 567–581. https://doi.org/10.1518/hfes.46.3.567.50405
31.
KareevY.ArnonS.Horwitz-ZeligerR. (2002). On the misperception of variability. Journal of Experimental Psychology: General, 131(2), 287–297. https://doi.org/10.1037/0096-3445.131.2.287
LeeJ. D.MorayN. (1992). Operators’ reliance on automation during the control of a semi-automatic process. Proceedings 1992 IEEE International Conference on Systems, Man, and Cybernetics, 1(1), 526–530. https://doi.org/10.1109/ICSMC.1992.271720
34.
MadhavanP.WiegmannD. A. (2007). Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Human Factors, 49(5), 773–785. https://doi.org/10.1518/001872007X230154
35.
MadhavanP.WiegmannD. A.LacsonF. C. (2006). Automation failures on tasks easily performed by operators undermine trust in automated aids. Human Factors, 48(2), 241–256. https://doi.org/10.1518/001872006777724408
36.
MerrittS. M.UnnerstallJ. L.LeeD.HuberK. (2015). Measuring individual differences in the perfect automation schema. Human Factors, 57(5), 740–753. https://doi.org/10.1177/0018720815581247
37.
MeyerJ. (2001). Effects of warning validity and proximity on responses to warnings. Human Factors: The Journal of the Human Factors and Ergonomics Society, 43(4), 563–572. https://doi.org/10.1518/001872001775870395
38.
MeyerJ.WiczorekR.GünzlerT. (2014). Measures of reliance and compliance in aided visual scanning. Human Factors: The Journal of the Human Factors and Ergonomics Society, 56(5), 840–849. https://doi.org/10.1177/0018720813512865
39.
MillerC. M.PattonC. E.EscobdeoA. R.Guzman-BonillaE. (2025). Whose wisdom? Human biases for decision support system source and scale. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 69(1), 2032–2037. https://doi.org/10.1177/10711813251360707
40.
MosierK. L.SkitkaL. J. (1999). Automation use and automation bias. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 43(3), 344–348. https://doi.org/10.1177/154193129904300346
41.
ObrechtN. A.ChapmanG. B.GelmanR. (2007). Intuitivet tests: Lay use of statistical information. Psychonomic Bulletin & Review, 14(6), 1147–1152. https://doi.org/10.3758/BF03193104
PattonC. E.WickensC. D.SmithC. A. P.NobleK. M.CleggB. A. (2023). Supporting detection of hostile intentions: Automated assistance in a dynamic decision-making context. Cognitive Research: Principles and Implications, 8(1), 69. https://doi.org/10.1186/s41235-023-00519-5
45.
PetersonC. R.BeachL. R. (1967). Man as an intuitive statistician. Psychological Bulletin, 68(1), 29–46. https://doi.org/10.1037/h0024722
46.
PharmerR. L.WickensC. D.CleggB. A. (2022). When transparency fails: Compliance with decision support in nautical collision avoidance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 397–401. https://doi.org/10.1177/1071181322661321
47.
PollardP. (1984). Intuitive judgments of proportions, means, and variances: A review. Current Psychological Research & Reviews, 3(1), 5–18. https://doi.org/10.1007/BF02686528
48.
PopV. L.ShrewsburyA.DursoF. T. (2015). Individual differences in the calibration of trust in automation. Human Factors, 57(4), 545–556. https://doi.org/10.1177/0018720814564422
49.
RiceS. (2009). Examining single- and multiple-process theories of trust in automation. The Journal of General Psychology, 136(3), 303–322. https://doi.org/10.3200/GENP.136.3.303-322
50.
RiceS.McCarleyJ. S. (2011). Effects of response bias and judgment framing on operator use of an automated aid in a target detection task. Journal of Experimental Psychology: Applied, 17(4), 320–331. https://doi.org/10.1037/a0024243
51.
RiegerT.ManzeyD. (2022). Human performance consequences of automated decision aids: The impact of time pressure. Human Factors, 64(4), 617–634. https://doi.org/10.1177/0018720820965019
52.
RossJ. M.SzalmaJ. L.HancockP. A.BarnettJ. S.TaylorG. (2008). The effect of automation reliability on user automation trust and reliance in a search-and-rescue scenario. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 52(19), 1340–1344. https://doi.org/10.1177/154193120805201908
53.
RoviraE.McGarryK.ParasuramanR. (2007). Effects of imperfect automation on decision making in a simulated command and control task. Human Factors, 49(1), 76–87. https://doi.org/10.1518/001872007779598082
54.
RoviraE.ParasuramanR. (2010). Transitioning to future air traffic management: Effects of imperfect automation on controller attention and performance. Human Factors, 52(3), 411–425. https://doi.org/10.1177/0018720810375692
55.
SanchezJ. (2006). Factors that affect trust and reliance on an automated aid. Georgia Institute of Technology.
56.
SeppeltB. D.LeeJ. D. (2019). Keeping the driver in the loop: Dynamic feedback to support appropriate use of imperfect vehicle control automation. International Journal of Human-Computer Studies, 125(1), 66–80. https://doi.org/10.1016/j.ijhcs.2018.12.009
57.
StantonN. S.RagsdaleS. A.BustamanteE. A. (2009). The effects of system technology and probability type on trust, compliance, and reliance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 53(18), 1368–1372. https://doi.org/10.1177/154193120905301843
58.
StevensS. S.GalanterE. H. (1957). Ratio scales and category scales for a dozen perceptual continua. Journal of Experimental Psychology, 54(6), 377–411. https://doi.org/10.1037/h0043680
59.
StricklandL.FarrellS.WilsonM. K.HutchinsonJ.LoftS. (2024). How do humans learn about the reliability of automation?Cognitive Research: Principles and Implications, 9(1), 8. https://doi.org/10.1186/s41235-024-00533-1
WardenA. C.WittJ. K.FuM.DoddM. D. (2025). Overestimation of variability in ensembles of color value and size. Attention, Perception, & Psychophysics, 87(5), 1579–1603. https://doi.org/10.3758/s13414-025-03098-3
WickensC. D.CleggB. A.WittJ. K.SmithC. A. P.HerdenerN.SpahrK. S. (2020). Model of variability estimation: Factors influencing human prediction and estimation of variability in continuous information. Theoretical Issues in Ergonomics Science, 21(2), 220–238. https://doi.org/10.1080/1463922X.2019.1679907
66.
WickensC. D.DixonS. R. (2007). The benefits of imperfect diagnostic automation: A synthesis of the literature. Theoretical Issues in Ergonomics Science, 8(3), 201–212. https://doi.org/10.1080/14639220500370105
67.
WickensC. D.HeltonW. S.HollandsJ. G.BanburyS. (2021). Engineering psychology and human performance (5th ed.). Routledge. https://doi.org/10.4324/9781003177616
68.
WiegmannD. A.RichA.ZhangH. (2001). Automated diagnostic aids: The effects of aid reliability on users’ trust and reliance. Theoretical Issues in Ergonomics Science, 2(4), 352–367. https://doi.org/10.1080/14639220110110306
69.
WoznyD. R.BeierholmU. R.ShamsL. (2010). Probability matching as a computational strategy used in perception. PLoS Computational Biology, 6(8), Article e1000871. https://doi.org/10.1371/journal.pcbi.1000871
70.
YangX. J.SchemanskeC.SearleC. (2023). Toward quantifying trust dynamics: How people adjust their trust after moment-to-moment interaction with automation. Human Factors, 65(5), 862–878.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
