It is often assumed that two heads are better than one, but reliance on decision aids is often inappropriate. Decisions to rely on an aid are thought to be based on a comparison between the perceived reliability of own performance and that of the decision aid. Unfortunately, perceived reliabilities are unlikely to be perfectly calibrated. This may result in inappropriate decisions to rely on advice. In a laboratory experiment with 40 participants, we studied whether calibration improves after practice, whether calibration of own reliability differs from calibration of the aid's reliability and whether unreliability of the aid is attributed differently. Under-trust in own reliability disappears after practice but under-trust in the aid's reliability persists. Unreliability of the decision aid is less likely to be attributed to temporary, external and uncontrollable factors. This asymmetry in attribution and calibration may explain under-reliance on decision aids.
Get full access to this article
View all access options for this article.
References
1.
AlbaJ. W.HutchinsonJ. W. (2000). Knowledge calibration: What consumers know and what they think they know. Journal of Consumer Research, 27, 123–156.
2.
BrennerL. A.KoehlerD. J.LibermanV.TverskyA. (1996). Overconfidence in probability and frequency judgments: A critical examination. Organizational Behavior and Human Decision Processes, 65, 212–219.
3.
CohenM. S. (2000). A situation-specific model of trust in decision aids. In Proceedings of Human Performance, Situation Awareness & Automation: User-Centered Design for the New Millennium, Savannah, GA, 143–148.
4.
van DongenK.van MaanenP.-P. (2005). Designing for Dynamic Task Allocation. In Proceedings of the Seventh International Naturalistic Decision Making Conference (NDM7), Amsterdam, The Netherlands.
5.
van DongenK.SchraagenJ. M.EikelboomA.BrakeG.te (2005). Supporting Decision Making by a Critical Thinking Tool. In Proceedings of the 49th Annual Meeting of the Human Factors and Ergonomics Society.Santa Monica: CA. Human Factors and Ergonomics Society, 517–521.
6.
DzindoletM. T.PierceL. G.BeckH. P.DaweL. A. (1999). Misuse and disuse of automated aids. In Proceedings of the Human Factors Society 43rd Annual MeetingSanta Monica, CA: Human Factors and Ergonomics Society, 339–343.
7.
DzindoletM. T.PetersonS. A.PomrankyR. A.PierceL. G.BeckH. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58, 697–718.
8.
FalconeR.CastelfranchiC. (2004). Trust dynamics: How trust is influenced by direct experiences and by trust itself. In Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2004), New York, USA, 740–747.
9.
GilbertD. T.MaloneP. S. (1995). The correspondence bias. Psychological Bulletin, 117, 21–38.
10.
HarveyN.FischerI. (1997). Taking advice: Accepting help, Improving judgment, and Sharing responsibility. Organizational Behavior and Human Decision Processes, 70, 117–133.
11.
KlaymanJ.SollJ. B.Gonzalez-VallejoC.BarlasS. (1999). Overconfidence: It Depends on How, What, and Who You Ask. Organizational Behavior and Human Decision Processes, 79, 216–247.
12.
LeeJ. D.MorayN. (1992). Trust, control strategies and allocation of function in human machine systems. Ergonomics, 22, 671–691.
13.
LeeJ. D.SeeK. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46, 50–80.
14.
van MaanenP.-P.van DongenK. (2005). Towards Task Allocation Decision Support by means of Cognitive Modeling of Trust. In Proceedings of the Eighth International Workshop on Trust in Agent Societies (Trust 2005), Utrecht, The Netherlands, 168–77.
15.
MadhavanP.WiegmannD. A. (2005). Effects of information source, pedigree, and reliability on operators' utilization of diagnostic advice. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting, 487–491.
16.
MadhavanP.WiegmannD. A.LacsonF. C. (2003). Automation failures on tasks easily performed by operators undermines trust in automated aids. In Proceedings of the 47th Annual Meeting of the Human Factors and Ergonomics Society.Santa Monica: CA. Human Factors and Ergonomics Society.
17.
MayR. S. (1987). Calibration of subjective probabilities: A cognitive analysis of inference processes in overconfidence (in German). Frankfurt: Peter Lang.
18.
MayR. S. (1988). Overconfidence in overconfidence. In ChaikanA.KindlerJ.KissI. (Eds.), Proceedings of the 4th FUR Conference.Dordrecht: Kluwer.
19.
MorayN.InagakiT.ItohM. (2000). Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied, 6, 44–58.
20.
ParasuramanR.RileyV.A. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39, 230–253.
21.
SniezekJ. A.van SwolL. M. (2001). Trust, confidence, and expertise in a judge–advisor system. Organizational Behavior and Human Decision Processes, 84, 288–307.
22.
VreugdenhilH.KoeleP. (1988). Underconfidence in predicting future events. Bulletin of the Psychonomic Society, 26, 236–237.
23.
WiegmannD. A.RichA.ZhangH. (2001). Automated diagnostic aids: The effects of aid reliability on user' trust and reliance. Theoretical Issues in Ergonomics Science, 2, 352–367.
24.
YanivI. (2004). Receiving other people's advice: Influence and benefit. Organizational Behavior and Human Decision Processes, 93, 1–13.
