In this study, we investigated strategies to address trust issues arising from errors in large language models (LLMs). The study examined the impact of confidence scores, system capability explanations, and user feedback on trust restoration post-error. 68 participants viewed the responses of an LLM to 20 general trivia questions, with an error introduced on the third trial. Each participant was presented with one mitigation strategy. Participants rated their overall trust in the model and the reliability of the answer. Results showed an immediate drop in trust after the error; however, there were no differences across the three strategies in trust recovery. All conditions had a logarithmic trend in trust recovery following error. Differences in overall trust were predicted by perceived reliability of the answer, suggesting that participants were evaluating results critically and using that to inform their trust in the model. Qualitative data supported this finding; participants expressed lasting distrust despite the LLM’s later accuracy. Results showcase the need to prioritize accuracy in LLM deployment, because early errors may irrevocably damage user trust calibration and later adoption.
AroyoA. M.de BruyneJ.DheuO.Fosch-VillarongaE.GudkovA.HochH.JonesS.LutzC.SætraH.SolbergM.Tamò-LarrieuxA. (2021). Overtrusting robots: Setting a research agenda to mitigate overtrust in automation. Paladyn, Journal of Behavioral Robotics, 12(1), 423–436. https://doi.org/10.1515/pjbr-2021-0029
2.
BansalG.NushiB.KamarE.LaseckiW. S.WeldD. S.HorvitzE. (2019, October). Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI conference on human computation and crowdsourcing, 7, 2–11. https://doi.org/10.1609/hcomp.v7i1.5285
3.
BuçincaZ.MalayaM. B.GajosK. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of ACM on Human-Computer Interaction, 5(CSCW1), 1–21. https://doi.org/10.1145/3449287
4.
BurtonJ. W.SteinM.-K.JensenT. B. (2020). A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making, 33(2), 220–239. https://doi.org/10.1002/bdm.2155
5.
CardonP. W.GetchellK.CarradiniS.FleischmannC.StappJ. (2023). Generative AI in the workplace: Employee perspectives of ChatGPT benefits and organizational policies. https://osf.io/preprints/socarxiv/b3ezy
ColarelliS. M.ThompsonM. (2008). Stubborn reliance on human nature in employee selection: Statistical decision aids are evolutionarily novel. Industrial and Organizational Psychology, 1(3), 347–351. https://doi.org/10.1111/j.1754-9434.2008.00060.x
8.
CooperG. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444–452. https://doi.org/10.1007/s10956-023-10039-y
9.
De AngelisL.BaglivoF.ArzilliG.PriviteraG. P.FerraginaP.TozziA. E.RizzoC. (2023). ChatGPT and the rise of large language models: The new AI-driven infodemic threat in public health. Frontiers in Public Health, 11, 1166120. https://doi.org/10.3389/fpubh.2023.1166120
10.
de VisserE. J.PakR.ShawT. H. (2018). From ‘automation’ to ‘autonomy’: The importance of trust repair in human–machine interaction. Ergonomics, 61(10), 1409–1427. https://doi.org/10.1080/00140139.2018.1457725
11.
DietvorstB. J.SimmonsJ. P.MasseyC. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033
12.
DietvorstB. J.SimmonsJ. P.MasseyC. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155–1170. https://doi.org/10.1287/mnsc.2016.2643
DzindoletM. T.PetersonS. A.PomrankyR. A.PierceL. G.BeckH. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58(6), 697–718. https://doi.org/10.1016/s1071-5819(03)00038-7
15.
GliksonE.WoolleyA. W. (2020). Human trust in artificial intelligence: Review of empirical research. The Academy of Management Annals, 14(2), 627–660. https://doi.org/10.5465/annals.2018.0057
16.
GoebelR.ChanderA.HolzingerK.LecueF.AkataZ.StumpfS.KiesebergP.HolzingerA. (2018). Explainable AI: The new 42? In HolzingerA.KiesebergP.TjoaA. M.WeipplE. (Eds.), Machine learning and knowledge extraction (pp. 295–303). Springer International Publishing.
17.
HoffK. A.BashirM. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570
18.
HuangL.YuW.MaW.ZhongW.FengZ.WangH.ChenQ.PengW.FengX.QinB.LiuT. (2024). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. https://doi.org/10.1145/3703155
KaurH.NoriH.JenkinsS.CaruanaR.WallachH.Wortman VaughanJ. (2020). Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems,Honolulu, HI, USA, 2020, 1–14. https://doi.org/10.1145/3313831.3376219
21.
KennyE. M.FordC.QuinnM.KeaneM. T. (2021). Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in XAI user studies. Artificial Intelligence, 294, 103459. https://doi.org/10.1016/j.artint.2021.103459
22.
KimP. H.DirksK. T.CooperC. D. (2009). The repair of trust: A dynamic bilateral perspective and multilevel conceptualization. Academy of Management Review, 34(3), 401–422. https://doi.org/10.5465/amr.2009.40631887
23.
KimT.SongH. (2021). How should intelligent agents apologize to restore trust? Interaction effects between anthropomorphism and apology attribution on trust repair. Telematics and Informatics, 61, 101595. https://doi.org/10.1016/j.tele.2021.101595
24.
KöbisN.MossinkL. D. (2021). Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry. Computers in Human Behavior, 114, 106553. https://doi.org/10.1016/j.chb.2020.106553
25.
KohnS. C.de VisserE. J.WieseE.LeeY. C.ShawT. H. (2021). Measurement of trust in automation: A narrative review and reference guide. Frontiers in Psychology, 12, 604977. https://doi.org/10.3389/fpsyg.2021.604977
26.
KramerR. M.LewickiR. J. (2010). Repairing and enhancing trust: Approaches to reducing organizational trust deficits. The Academy of Management Annals, 4(1), 245–277. https://doi.org/10.1080/19416520.2010.487403
27.
LeeJ.MorayN. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243–1270. https://doi.org/10.1080/00140139208967392
28.
LeeJ. D.MorayN. (1994). Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies, 40(1), 153–184. https://doi.org/10.1006/ijhc.1994.1007
LiJ.ChenJ.RenR.ChengX.ZhaoW. X.NieJ.-Y.WenJ.-R. (2024). The dawn after the dark: An empirical study on factuality hallucination in large language models. arXiv preprint arXiv:2401.03205.
31.
MadsenM.GregorS. (2000). Measuring human-computer trust. 11th australasian conference on information systems. (53, pp. 6–8). Citeseer.
32.
MahmudH.IslamA. N.AhmedS. I.SmolanderK. (2022). What influences algorithmic decision-making? A systematic literature review on algorithm aversion. Technological Forecasting and Social Change, 175, 121390. https://doi.org/10.1016/j.techfore.2021.121390
33.
MayerR. C.DavisJ. H.SchoormanF. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. https://doi.org/10.2307/258792
34.
McGuirlJ. M.SarterN. B. (2006). Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information. Human Factors, 48(4), 656–665. https://doi.org/10.1518/001872006779166334
35.
MerrittS. M.UnnerstallJ. L.LeeD.HuberK. (2015). Measuring individual differences in the perfect automation schema. Human Factors, 57(5), 740–753. https://doi.org/10.1177/0018720815581247.
36.
MittalS.RaineyL. (2015, July). Harnessing emergence: The control and design of emergent behavior in system of systems engineering. In Proceedings of the conference on summer computer simulation, Chicago, Illinois, 2015. (pp. 1-10).
37.
MuirB. M.MorayN. (1996). Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39(3), 429–460. https://doi.org/10.1080/00140139608964474
38.
PopV. L.ShrewsburyA.DursoF. T. (2015). Individual differences in the calibration of trust in automation. Human Factors, 57(4), 545–556. https://doi.org/10.1177/0018720814564422
39.
RibeiroM. T.SinghS.GuestrinC. (2016, August). “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, California, USA, 2016, 1135–1144. https://doi.org/10.1145/2939672.2939778
40.
RobinetteP.HowardA. M.WagnerA. R. (2015). Timing is key for robot trust repair. Social Robotics.
41.
SchererL. D.de VriesM.Zikmund-FisherB. J.WittemanH. O.FagerlinA. (2015). Trust in deliberation: The consequences of deliberative decision strategies for medical decisions. Health Psychology, 34(11), 1090–1099. https://doi.org/10.1037/hea0000203
42.
SharanN. N.RomanoD. M. (2020). The effects of personality and locus of control on trust in humans versus artificial intelligence. Heliyon, 6(8), Article e04572. https://doi.org/10.1016/j.heliyon.2020.e04572
ThirunavukarasuA. J.TingD. S. J.ElangovanK.GutierrezL.TanT. F.TingD. S. W. (2023). Large language models in medicine. Nature Medicine, 29(8), 1930–1940. https://doi.org/10.1038/s41591-023-02448-8
45.
TomlinsonE. C.MryerR. C. (2009). The role of causal attribution dimensions in trust repair. Academy of Management Review, 34(1), 85–104. https://doi.org/10.5465/amr.2009.35713291
46.
ToreiniE.AitkenM.CoopamootooK.ElliottK.ZelayaC. G.van MoorselA. (2020, January). The relationship between trust in AI and trustworthy machine learning technologies. In Proceedings of the 2020 conference on fairness, accountability, and transparency, Barcelona, Spain, 2020, 272–283. https://doi.org/10.1145/3351095.3372834
47.
van der WaaJ.NieuwburgE.CremersA.NeerincxM. (2021). Evaluating XAI: A comparison of rule-based and example-based explanations. Artificial Intelligence, 291, 103404. https://doi.org/10.1016/j.artint.2020.103404
48.
VigJ. (2019, May). BertViz: A tool for visualizing multihead self-attention in the BERT model. In ICLR workshop: Debugging machine learning models, New Orleans, LA, 2019. (Vol. 3, p. 1-6).
YangF.HuangZ.ScholtzJ.ArendtD. L. (2020, March). How do visual explanations foster end users’ appropriate trust in machine learning? In Proceedings of the 25th international conference on intelligent user interfaces, Cagliari, Italy, 2020, 189–201. https://doi.org/10.1145/3377325.3377480
51.
YinM.VaughanJ. W.WallachH. (2019, May). Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 CHI conference on human factors in computing systems, Glasgow, Scotland Uk, 2019. https://doi.org/10.1145/3290605.3300509. (pp. 1-12).
52.
YuK.BerkovskyS.TaibR.ZhouJ.ChenF. (2019). Do I trust my machine teammate? An investigation from perception to decision. Proceedings of the 24th international conference on intelligent user interfaces, Marina del Rey, California, Intelligent User Interfaces. 2019. https://doi.org/10.1145/3301275.330227. (pp. 460-468).
ZhangY.LiaoQ. V.BellamyR. K. E. (2020, January). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency, Barcelona, Spain, 2020. https://doi.org/10.1145/3351095.3372852 (pp. 295-305).
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
