Sage Journals: Discover world-class research

Abstract

There is currently significant research and industry interest in engineering machines and algorithms that humans will trust. This is justified as a means for facilitating the adoption of developing technology. However, there are many problems with trust that directly relate to its epistemological validity, usefulness, ethical implications, and potential for human disempowerment. This article explores trust from this perspective in the hopes of encouraging the human factors engineering community to de-emphasize trust as an end goal and replace it with more objective measures and good human factors engineering practices.

Keywords

trust human-machine interaction human-automation interaction psychometrics validity artificial intelligence machine learning

Get full access to this article

View all access options for this article.

References

Air Force Research Laboratory, State University of New York, IBM, NYSTEC, & National Security Innovation Network . (2021). Trusted AI challenge series. Innovare Advancement Center. https://assets.website-files.com/5f47f05cf743023a854e9982/60885a1587a12f5393a33f67_TAITopic3RFPNSIN_NYSTECfinal26Apr.pdf

Annett

(2002). Subjective rating scales: Science or art? Ergonomics, 45(14), 966–987. https://doi.org/10.1080/00140130210166951

Bainbridge

(1983). Ironies of automation. Automatica, 19(6), 775–780. https://doi.org/10.1016/0005-1098(83)90046-8

Biran

Cotton

(2017). Explanation and justification in machine learning: A survey. In IJCAI-17 workshop on explainable AI (XAI) (8, pp. 8–13).

Bolton

M. L.

Biltekoff

Humphrey

(n.d.-a). The level of measurement of subjective situation awareness and its dimension in the situation awareness rating technique (SART). IEEE Transactions on Human-Machine Systems. (In Press). https://doi.org/10.1109/THMS.2021.3121960

Bolton

M. L.

Biltekoff

Humphrey

(n.d.-b). The level of measurement of the NASA task load index and its constituent dimensions. In IEEE transactions on human-machine systems, 10 pages. (Under Review).

Braga

D. D. S.

Niemann

Hellingrath

Neto

F. B. D. L.

(2018). Survey on computational trust and reputation models. ACM Computing Surveys (CSUR), 51(5), 1–40. https://doi.org/10.1145/3236008

Braucher

Orbach

(2015). Scamming: The misunderstood confidence man. Yale Journal of Law and the Humanities, 27(2015), 249. https://doi.org/10.2139/ssrn.2314071

Chancey

E. T.

Bliss

J. P.

Yamani

Handley

H. A.

(2017). Trust and the compliance–reliance paradigm: The effects of risk, error bias, and reliability on trust and dependence. Human Factors, 59(3), 333–345. https://doi.org/10.1177/0018720816682648

10.

Cho

J.-H.

Chan

Adali

(2015). A survey on trust modeling. ACM Computing Surveys (CSUR), 48(2), 1–40. https://doi.org/10.1145/2815595

11.

Cronk

T. M.

(June 22, 2021). Hicks announces new artificial intelligence initiative. DOD News. https://www.defense.gov/News/News-Stories/Article/Article/2667212/hicks-announces-new-artificial-intelligence-initiative/

12.

Dekker

Hollnagel

(2004). Human factors and folk models. Cognition, Technology & Work, 6(2), 79–86. https://doi.org/10.1007/s10111-003-0136-9

13.

Dekker

Nyce

J. M.

(2015). From figments to figures: Ontological alchemy in human factors research. Cognition, Technology & Work, 17(2), 185–187. https://doi.org/10.1007/s10111-015-0321-7

14.

Delbufalo

(2015). Subjective trust and perceived risk influences on exchange performance in supplier–manufacturer relationships. Scandinavian Journal of Management, 31(1), 84–101. https://doi.org/10.1016/j.scaman.2014.06.002

15.

Earle

Siegrist

(2008). Trust, confidence and cooperation model: A framework for understanding the relation between trust and risk perception. International Journal of Global Environmental Issues, 8(1–2), 17–29. https://doi.org/10.1504/IJGENVI.2008.017257

16.

Earle

T. C.

(2010). Trust in risk management: A model-based review of empirical research. Risk Analysis: An International Journal, 30(4), 541–574. https://doi.org/10.1111/j.1539-6924.2010.01398.x

17.

Eignor

D. R.

(2013). The standards for educational and psychological testing. American Psychological Association.

18.

European Union Aviation Safety Agency (2021). EASA concept paper: First usable guidance for level 1 machine learning applications: A deliverable of the EASA AI roadmap (tech. Rep. No. Proposed issue 01). European Union Aviation Safety Agency.

19.

Gebru

Zeleke

Blankson

Nabil

Nateghi

Homaifar

Tunstel

(2022). A review on human–machine trust evaluation: Human-centric and machine-centric perspectives. IEEE Transactions on Human-Machine Systems, 52(5), 952–962. https://doi.org/10.1109/THMS.2022.3144956

20.

Goldberg

(May 13, 2016). Lack of trust in internet privacy and security may deter economic and other online activities. National Telecommunications and Information Administration, United States Department of Commerce. https://www.ntia.doc.gov/blog/2016/lack-trust-internet-privacy-and-security-may-deter-economic-and-other-online-activities

21.

Halpin

S. M.

Johnson

E. M.

Thornberry

J. A.

(1973). Cognitive reliability in manned systems. IEEE Transactions on Reliability, 22(3), 165–170. https://doi.org/10.1109/TR.1973.5215932

22.

Hancock

P. A.

Billings

D. R.

Schaefer

K. E.

Chen

J. Y.

De Visser

E. J.

Parasuraman

(2011). A meta-analysis of factors affecting trust in human-robot interaction. Human Factors, 53(5), 517–527. https://doi.org/10.1177/0018720811417254

23.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (task load index): Results of empirical and theoretical research. Advances in Psychology, 52, 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

24.

Helbing

Frey

B. S.

Gigerenzer

Hafen

Hagner

Hofstetter

van den Hoven

Zicari

R. V.

Zwitter

(2019). Will democracy survive big data and artificial intelligence? In Towards digital enlightenment (pp. 73–98). Springer.

25.

Hoff

K. A.

Bashir

(2015). Trust in automation integrating empirical evidence on factors that influence trust. Human Factors, 57(3), 407–434. https://doi.org/10.1177/0018720814547570

26.

Israelsen

B. W.

Ahmed

N. R.

(2019). dave... I can assure you... that it’s going to be all right…” A definition, case for, and survey of algorithmic assurances in human-autonomy trust relationships. ACM Computing Surveys (CSUR), 51(6), 1–37. https://doi.org/10.1145/3267338

27.

James

H. S

Jr (2002). The trust paradox: A survey of economic inquiries into the nature of trust and trustworthiness. Journal of Economic Behavior & Organization, 47(3), 291–307. https://doi.org/10.1016/S0167-2681(01)00214-1

28.

Jian

J.-Y.

Bisantz

A. M.

Drury

C. G.

(2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71. https://doi.org/10.1207/S15327566IJCE0401_04

29.

Lee

J. D.

Moray

(1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243–1270. https://doi.org/10.1080/00140139208967392

30.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392

31.

Levine

T. R.

(2014). Encyclopedia of deception Vol. (2). Sage Publications.

32.

Llinas

(2022). Putting AI in the critical loop: Assured trust and autonomy in human-machine teams. In Association for the advancement of artificial intelligence. https://aaai.org/Symposia/Spring/sss22symposia.php

33.

Lyons

J. B.

Stokes

C. K.

(2012). Human–human reliance in the context of automation. Human Factors, 54(1), 112–121. https://doi.org/10.1177/0018720811427034

34.

Madsen

Gregor

(2000). Measuring human-computer trust. In 11th australasian conference on information systems (Vol. 53, pp. 6–8). Springer.

35.

Matthews

De Winter

Hancock

P. A.

(2020). What do subjective workload scales really measure? Operational and representational solutions to divergence of workload measures. Theoretical Issues in Ergonomics Science, 21(4), 369–396. https://doi.org/10.1080/1463922X.2018.1547459

36.

Mayer

R. C.

Davis

J. H.

Schoorman

F. D.

(1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. https://doi.org/10.2307/258792

37.

Meyer

(2004). Conceptual issues in the study of dynamic hazard warnings. Human Factors, 46(2), 196–204. https://doi.org/10.1518/hfes.46.2.196.37335

38.

Meyer

Lee

J. D.

(2013). Trust, reliance, and compliance. In Lee

J. D.

Kirlik

(Eds.), The oxford handbook of cognitive engineering. Oxford University Press.

39.

National Science Foundation, Department of Homeland Security, Science & Technology Directorate, Institute of Education Sciences, U.S. Department of Education, National Institute of Food and Agriculture, National Institute of Standards and Technology, Department of Defense, & IBM Corp . (2020). National artificial intelligence (AI) research institutes: Accelerating research, transforming society, and growing the American workforce: Program solicitation (tech. Rep. No. NSF 22-502). National Science Foundation.

40.

National Institute of Standards and Technology . (2019). US leadership in AI: A plan for federal engagement in developing technical standards and related tools. Prepared in response to executive order 13859 (tech. Rep.). US Department of Commerce Washington, DC. https://www.nist.gov/system/files/documents/2019/08/10/ai_standards_fedengagement_plan_9aug2019.pdf

41.

National Science Foundation, National Institute of Food and Agriculture, Department of Homeland Security, Science & Technology Directorate, U.S. Department of Transportation, Federal Highway Administration, & U.S. Department of Veterans Affairs . (2020). National artificial intelligence (AI) research institutes: Accelerating research, transforming society, and growing the American workforce: Program solicitation (tech. Rep. No. NSF 20-503). National Science Foundation.

42.

National Society of Professional Engineers . (2019). NSPE code of ethics for engineers. National Society of Professional Engineers. https://www.nspe.org/sites/default/files/resources/pdfs/Ethics/CodeofEthics/NSPECodeofEthicsforEngineers.pdf

43.

Orbach

Huang

(2018). Con men and their enablers: The anatomy of confidence games. Social Research: An International Quarterly, 85(4), 795–822. https://doi.org/10.1353/sor.2018.0050

44.

Parasuraman

Riley

(1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230. https://doi.org/10.1518/001872097778543886

45.

Parasuraman

Sheridan

T. B.

Wickens

C. D.

(2008). Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2(2), 140–160. https://doi.org/10.1518/155534308X284417

46.

Rousseau

D. M.

Sitkin

S. B.

Burt

R. S.

Camerer

(1998). Not so different after all: A cross-discipline view of trust. Academy of Management Review, 23(3), 393–404. https://doi.org/10.5465/amr.1998.926617

47.

Schaefer

K. E.

Chen

J. Y.

Szalma

J. L.

Hancock

P. A.

(2016). A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors, 58(3), 377–400. https://doi.org/10.1177/0018720816634228

48.

Schmidt

(April 2018). Statement of Dr. Eric Schmidt: House armed services committee (Vol. 17). House Armed Services Committee. https://docs.house.gov/meetings/AS/AS00/20180417/108132/HHRG-115-AS00-Wstate-SchmidtE-20180417.pdf

49.

Shahrdar

Menezes

Nojoumian

(2018). A survey on trust in autonomous systems. In Science and information conference (pp. 368–386).

50.

Sheridan

T. B.

Ferrell

W. R.

(1974). Man-machine systems; information, control, and decision models of human performance. the MIT press.

51.

Siegrist

(2021). Trust and risk perception: A critical review of the literature. Risk analysis, 41(3), 480–490. https://doi.org/10.1111/risa.13325

52.

Summit on trusted autonomy research and technology: Agenda (2022). West Lafayette: Purdue University.

53.

Vashitz

Meyer

Parmet

Peleg

Goldfarb

Porath

Gilutz

(2009). Defining and measuring physicians’ responses to clinical reminders. Journal of Biomedical Informatics, 42(2), 317–326. https://doi.org/10.1016/j.jbi.2008.10.001

54.

Wang

Bisantz

A. M.

Bolton

M. L.

Cavuoto

Chandola

(2020). Explaining supervised learning models: A preliminary study on binary classifiers. Ergonomics in Design, 28(3), 20–26. https://doi.org/10.1177/1064804620901641

55.

Wei

Bolton

M. L.

Humphrey

(2019). Subjective measurement of trust: Is it on the level? In Proceedings of the human factors and ergonomics society annual meeting (pp. 212–216). Sage.

56.

Wei

Bolton

M. L.

Humphrey

(2020). The level of measurement of trust in automation. Theoretical Issues in Ergonomics Science, 22(3), 274–295. https://doi.org/10.1080/1463922X.2020.1766596

57.

Yang

X. J.

Schemanske

Searle

(2021). Toward quantifying trust dynamics: How people adjust their trust after moment-to-moment interaction with automation. Human Factors.

58.

Zhou

Sun

Liu

Burnett

(2022). Factors affecting pedestrians’ trust in automated vehicles: Literature review and theoretical model. IEEE Transactions on Human-Machine Systems, 52(3), 490–500. https://doi.org/10.1109/THMS.2021.3112956

Trust is Not a Virtue: Why We Should Not Trust Trust

Abstract

Keywords

Get full access to this article

References