Autonomous cars (ACs) controlled by artificial intelligence are expected to play a significant role in transportation in the near future. This study investigated determinants of trust in ACs.
Background
Trust in ACs influences different variables, including the intention to adopt AC technology. Several studies on risk perception have verified that shared value determines trust in risk managers. Previous research has confirmed the effect of value similarity on trust in artificial intelligence. We focused on moral beliefs, specifically utilitarianism (belief in promoting a greater good) and deontology (belief in condemning deliberate harm), and tested the effects of shared moral beliefs on trust in ACs.
Method
We conducted three experiments (N = 128, 71, and 196, for each), adopting a thought experiment similar to the well-known trolley problem. We manipulated shared moral beliefs (shared vs. unshared) and driver (AC vs. human), providing participants with different moral dilemma scenarios. Trust in ACs was measured through a questionnaire.
Results
The results of Experiment 1 showed that shared utilitarian belief strongly influenced trust in ACs. In Experiment 2 and Experiment 3, however, we did not find statistical evidence that shared deontological belief had an effect on trust in ACs.
Conclusion
The results of the three experiments suggest that the effect of shared moral beliefs on trust varies depending on the values that ACs share with humans.
Application
To promote AC implementation, policymakers and developers need to understand which values are shared between ACs and humans to enhance trust in ACs.
AllumN. (2007). An empirical test of competing theories of hazard-related trust: The case of GM food. Risk Analysis, 27, 935–946.doi:10.1111/j.1539-6924.2007.00933.xhttp://www.ncbi.nlm.nih.gov/pubmed/17958502
2.
AwadE.DsouzaS.KimR.SchulzJ.HenrichJ.ShariffA.BonnefonJ. -F.RahwanI. (2018). The moral machine experiment. Nature, 563, 59–64.doi:10.1038/s41586-018-0637-6http://www.ncbi.nlm.nih.gov/pubmed/30356211
3.
BellerJ.HeesenM.VollrathM. (2013). Improving the driver–automation interaction: An approach using automation uncertainty. The Journal of the Human Factors and Ergonomics Society, 55, 1130–1141.
4.
BennettR.VijaygopalR.KottaszR. (2019). Willingness of people with mental health disabilities to travel in driverless vehicles. Journal of Transport & Health, 12, 1–12.doi:10.1016/j.jth.2018.11.005
5.
BenthamJ. (1967). An introduction to the principles of morals and legislation (S. Yamashita, Trans.). InSekiY. (Ed.), Great books of the world 38 (pp.69–210). Chuo Koron. (Original work published 1789).
6.
BergmannL. T.SchlichtL.MeixnerC.KönigP.PipaG.BoshammerS.StephanA. (2018). Autonomous vehicles require socio-political Acceptance-An empirical and philosophical perspective on the problem of moral decision making. Frontiers in Behavioral Neuroscience, 12, 31.doi:10.3389/fnbeh.2018.00031http://www.ncbi.nlm.nih.gov/pubmed/29541023
7.
BigmanY. E.GrayK. (2018). People are averse to machines making moral decisions. Cognition, 181, 21–34.doi:10.1016/j.cognition.2018.08.003http://www.ncbi.nlm.nih.gov/pubmed/30107256
8.
BonnefonJ. -F.ShariffA.RahwanI. (2016). The social dilemma of autonomous vehicles. Science, 352, 1573–1576.doi:10.1126/science.aaf2654http://www.ncbi.nlm.nih.gov/pubmed/27339987
9.
BrellT.PhilipsenR.ZiefleM. (2019). sCARy! Risk perceptions in autonomous driving: The influence of experience on perceived benefits and barriers. Risk Analysis, 39, 342–357.doi:10.1111/risa.13190http://www.ncbi.nlm.nih.gov/pubmed/30239015
10.
BuckleyL.KayeS. -A.PradhanA. K. (2018). Psychosocial factors associated with intended use of automated vehicles: A simulated driving study. Accident Analysis & Prevention, 115, 202–208.doi:10.1016/j.aap.2018.03.021http://www.ncbi.nlm.nih.gov/pubmed/29631216
11.
Cannon-BowersJ. A.SalasE.ConverseS. A. (1993). Shared mental models in expert team decision making. InCastellanN. J. (Ed.), Individual and group decision making: Current issues (pp.221–246). Erlbaum.
12.
ChoJ. -H.ChanK.AdaliS. (2015). A survey on trust modeling. ACM Computing Surveys, 48, 1–40.doi:10.1145/2815595
13.
CumminsD. D.CumminsR. C. (2012). Emotion and deliberative reasoning in moral judgment. Frontiers in Psychology, 3, 328.doi:10.3389/fpsyg.2012.00328http://www.ncbi.nlm.nih.gov/pubmed/22973255
14.
de VriesP.MiddenC.BouwhuisD. (2003). The effects of errors on system trust, self-confidence, and the allocation of control in route planning. International Journal of Human-Computer Studies, 58, 719–735.doi:10.1016/S1071-5819(03)00039-9
15.
DecetyJ.CowellJ. M. (2015). Empathy, justice, and moral behavior. AJOB Neuroscience, 6, 3–14.doi:10.1080/21507740.2015.1047055http://www.ncbi.nlm.nih.gov/pubmed/26877887
16.
DewittB.FischhoffB.SahlinN. -E. (2019). ‘Moral machine’ experiment is no basis for policymaking. Nature, 567, 31.doi:10.1038/d41586-019-00766-xhttp://www.ncbi.nlm.nih.gov/pubmed/30837734
17.
DzindoletM. T.BeckH. P.PierceL. G.DaweL. A. (2001). A framework of automation use (Report No. ARL-TR-2412). Army Research Laboratory.
18.
EarleT. C.CvetkovichG. T. (1995). Social trust: Toward a cosmopolitan society. Praeger Press.
19.
FaulhaberA. K.DittmerA.BlindF.WächterM. A.TimmS.SütfeldL. R.StephanA.PipaG.KönigP. (2019). Human decisions in moral dilemmas are largely described by utilitarianism: Virtual car driving study provides guidelines for autonomous driving vehicles. Science and Engineering Ethics, 25, 399–418.doi:10.1007/s11948-018-0020-xhttp://www.ncbi.nlm.nih.gov/pubmed/29357047
20.
FootP. (1967). The problem of abortion and the doctrine of double effect. Oxford Review, 5, 5–15.
21.
FrisonA. K.WintersbergerP.RienerA.FrisonA. K.WintersbergerP.RienerA. (2016, October). First person trolley problem: Evaluation of drivers' ethical decisions in a driving simulator. Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, USA. 117–122.
22.
GoodallN. J. (2014). Ethical decision making during automated vehicle crashes. Transportation Research Record, 2424, 58–65.doi:10.3141/2424-07
23.
GrayK.WegnerD. M. (2009). Moral typecasting: Divergent perceptions of moral agents and moral patients. Journal of Personality and Social Psychology, 96, 505–520.doi:10.1037/a0013748http://www.ncbi.nlm.nih.gov/pubmed/19254100
24.
GrayK.YoungL.WaytzA. (2012). Mind perception is the essence of morality. Psychological Inquiry, 23, 101–124.doi:10.1080/1047840X.2012.651387http://www.ncbi.nlm.nih.gov/pubmed/22754268
25.
GreeneJ. D. (2007). Why are VMPFC patients more utilitarian? A dual-process theory of moral judgment explains. Trends in Cognitive Sciences, 11, 322–323.doi:10.1016/j.tics.2007.06.004http://www.ncbi.nlm.nih.gov/pubmed/17625951
26.
GreeneJ. D. (2016). Ethics. our driverless dilemma. Science, 352, 1514–1515.doi:10.1126/science.aaf9534http://www.ncbi.nlm.nih.gov/pubmed/27339966
27.
GreeneJ. D.MorelliS. A.LowenbergK.NystromL. E.CohenJ. D. (2008). Cognitive load selectively interferes with utilitarian moral judgment. Cognition, 107, 1144–1154.doi:10.1016/j.cognition.2007.11.004http://www.ncbi.nlm.nih.gov/pubmed/18158145
28.
GreeneJ. D.NystromL. E.EngellA. D.DarleyJ. M.CohenJ. D. (2004). The neural bases of cognitive conflict and control in moral judgment. Neuron, 44, 389–400.doi:10.1016/j.neuron.2004.09.027http://www.ncbi.nlm.nih.gov/pubmed/15473975
29.
GreeneJ. D.SommervilleR. B.NystromL. E.DarleyJ. M.CohenJ. D. (2001). An fMRI investigation of emotional engagement in moral judgment. Science, 293, 2105–2108.doi:10.1126/science.1062872http://www.ncbi.nlm.nih.gov/pubmed/11557895
30.
HergethS.LorenzL.VilimekR.KremsJ. F. (2016). Keep your scanners peeled: Gaze behavior as a measure of automation trust during highly automated driving. The Journal of the Human Factors and Ergonomics Society, 58, 509–519.doi:10.1177/0018720815625744http://www.ncbi.nlm.nih.gov/pubmed/26843570
31.
HjälmdahlM.KrupeniaS.ThorslundB. (2017). Driver behaviour and driver experience of partial and fully automated truck platooning – a simulator study. European Transport Research Review, 9, 8.doi:10.1007/s12544-017-0222-3
32.
HoffK. A.BashirM. (2015). Trust in automation: Integrating empirical evidence on factors that influence trust. The Journal of the Human Factors and Ergonomics Society, 57, 407–434.doi:10.1177/0018720814547570http://www.ncbi.nlm.nih.gov/pubmed/25875432
33.
HulseL. M.XieH.GaleaE. R. (2018). Perceptions of autonomous vehicles: Relationships with road users, risk, gender and age. Safety Science, 102, 1–13.doi:10.1016/j.ssci.2017.10.001
34.
JayaramanS. K.CreechC.Robert JrL. P.TilburyD. M.YangX. J.PradhanA. K.TsuiK. M. (2018, March). Trust in AC: An uncertainty reduction model of AC-pedestrian interactions. Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, USA. 133–134.
35.
JianJ. -Y.BisantzA. M.DruryC. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4, 53–71.doi:10.1207/S15327566IJCE0401_04
36.
KantI. (1976). Grundlegung zur metaphysic der sitten (H. Shinoda, Trans.). Iwanami Shoten. (Original work published 1785).
37.
KaurK.RampersadG. (2018). Trust in driverless cars: Investigating key factors influencing the adoption of driverless cars. Journal of Engineering and Technology Management, 48, 87–96.doi:10.1016/j.jengtecman.2018.04.006
38.
KörberM.BaselerE.BenglerK. (2018). Introduction matters: Manipulating trust in automation and reliance in automated driving. Applied Ergonomics, 66, 18–31.doi:10.1016/j.apergo.2017.07.006http://www.ncbi.nlm.nih.gov/pubmed/28958427
39.
LeeJ. -G.KimK. J.LeeS.ShinD. -H. (2015). Can autonomous vehicles be safe and trustworthy? Effects of appearance and autonomy of unmanned driving systems. International Journal of Human-Computer Interaction, 31, 682–691.doi:10.1080/10447318.2015.1070547
40.
LeeJ. D.SeeK. A. (2004). Trust in automation: Designing for appropriate reliance. The Journal of the Human Factors and Ergonomics Society, 46, 50–80.doi:10.1518/hfes.46.1.50.30392http://www.ncbi.nlm.nih.gov/pubmed/15151155
41.
LiS.ZhangJ.LiP.WangY.WangQ. (2019). Influencing factors of driving decision-making under the moral dilemma. IEEE Access, 7, 104132–104142.doi:10.1109/ACCESS.2019.2932043
42.
LiljamoT.LiimatainenH.PöllänenM. (2018). Attitudes and concerns on automated vehicles. Transportation Research Part F: Traffic Psychology and Behaviour, 59, 24–44.doi:10.1016/j.trf.2018.08.010
43.
LiuH.YangR.WangL.LiuP. (2019b). Evaluating initial public acceptance of highly and fully autonomous vehicles. International Journal of Human–Computer Interaction, 35, 919–931.doi:10.1080/10447318.2018.1561791
44.
LiuP.GuoQ.RenF.WangL.XuZ. (2019a). Willingness to pay for self-driving vehicles: Influences of demographic and psychological factors. Transportation Research Part C: Emerging Technologies, 100, 306–317.doi:10.1016/j.trc.2019.01.022
45.
LiuP.YangR.XuZ. (2019c). Public acceptance of fully automated driving: Effects of social trust and risk/benefit perceptions. Risk Analysis, 39, 326–341.doi:10.1111/risa.13143
46.
MadhavanP.WiegmannD. A. (2007). Similarities and differences between human–human and human–automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8, 277–301.doi:10.1080/14639220500337708
47.
MadsenM.GregorS. (2000). Measuring human-computer trust. InGableG.VitaleM. (Eds.), Proceedings of the 11th Australasian Conference on Information Systems (pp.53–64). AIS.
MayerR. C.DavisJ. H.SchoormanF. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20, 709–734.doi:10.5465/amr.1995.9508080335
50.
McDonaldM. M.DefeverA. M.NavarreteC. D. (2017). Killing for the greater good: Action aversion and the emotional inhibition of harm in moral dilemmas. Evolution and Human Behavior, 38, 770–778.doi:10.1016/j.evolhumbehav.2017.06.001
51.
MederB.FleischhutN.KrumnauN. -C.WaldmannM. R. (2019). How should autonomous cars drive? A preference for defaults in moral judgments under risk and uncertainty. Risk Analysis, 39, 295–314.doi:10.1111/risa.13178http://www.ncbi.nlm.nih.gov/pubmed/30157299
52.
MedinD.OjalehtoB.MarinA.BangM. (2017). Systems of (non-) diversity. Nature Human Behaviour, 1, 0088.doi:10.1038/s41562-017-0088
53.
MiuraA.KobayashiT. (2016). Survey satisficing inflates stereotypical responses in online experiment: The case of immigration study. Frontiers in Psychology, 7, 1563.doi:10.3389/fpsyg.2016.01563http://www.ncbi.nlm.nih.gov/pubmed/27803680
54.
MontagueE.XuJ.ChiouE. (2014). Shared experiences of technology and trust: An experimental study of physiological compliance between active and passive users in technology-mediated collaborative encounters. IEEE Transactions on Human-Machine Systems, 44, 614–624.doi:10.1109/THMS.2014.2325859
55.
NakayachiK.CvetkovichG. (2010). Public trust in government concerning tobacco control in Japan. Risk Analysis, 30, 143–152.doi:10.1111/j.1539-6924.2009.01306.xhttp://www.ncbi.nlm.nih.gov/pubmed/19878484
56.
NavarreteC. D.McDonaldM. M.MottM. L.AsherB. (2012). Virtual morality: Emotion and action in a simulated three-dimensional “trolley problem”. Emotion, 12, 364–370.doi:10.1037/a0025561http://www.ncbi.nlm.nih.gov/pubmed/22103331
57.
NeedhamM. D.VaskeJ. J. (2008). Hunter perceptions of similarity and trust in wildlife agencies and personal risk associated with chronic wasting disease. Society & Natural Resources, 21, 197–214.doi:10.1080/08941920701816336
NyholmS.SmidsJ. (2016). The ethics of accident-algorithms for self-driving cars: An applied trolley problem?Ethical Theory and Moral Practice, 19, 1275–1289.doi:10.1007/s10677-016-9745-2
OliveiraL.ProctorK.BurnsC. G.BirrellS. (2019). Driving style: How should an automated vehicle behave?Information, 10, 219.doi:10.3390/info10060219
63.
ParasuramanR.SheridanT. B.WickensC. D. (2008). Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2, 140–160.doi:10.1518/155534308X284417
64.
PayreW.CestacJ.DelhommeP. (2016). Fully automated driving: Impact of trust and practice on manual control recovery. The Journal of the Human Factors and Ergonomics Society, 58, 229–241.doi:10.1177/0018720815612319http://www.ncbi.nlm.nih.gov/pubmed/26646299
65.
PenmetsaP.AdanuE. K.WoodD.WangT.JonesS. L. (2019). Perceptions and expectations of autonomous vehicles–A snapshot of vulnerable road user opinion. Technological Forecasting and Social Change, 143, 9–13.doi:10.1016/j.techfore.2019.02.010
66.
PoortingaW.PidgeonN. F. (2003). Exploring the dimensionality of trust in risk regulation. Risk Analysis, 23, 961–972.doi:10.1111/1539-6924.00373http://www.ncbi.nlm.nih.gov/pubmed/12969411
67.
PugnettiC.SchläpferR. (2018). Customer preferences and implicit tradeoffs in accident scenarios for self-driving vehicle algorithms. Journal of Risk and Financial Management, 11, 28.doi:10.3390/jrfm11020028
68.
RaueM.D’AmbrosioL. A.WardC.LeeC.JacquillatC.CoughlinJ. F. (2019). The influence of feelings while driving regular cars on the perception and acceptance of self-driving cars. Risk Analysis, 39, 358–374.doi:10.1111/risa.13267http://www.ncbi.nlm.nih.gov/pubmed/30650211
69.
RousseauD. M.SitkinS. B.BurtR. S.CamererC. (1998). Not so different after all: A cross-discipline view of trust. Academy of Management Review, 23, 393–404.doi:10.5465/amr.1998.926617
70.
RuijtenP.TerkenJ.ChandramouliS. (2018). Enhancing trust in autonomous vehicles through intelligent user interfaces that mimic human behavior. Multimodal Technologies and Interaction, 2, 62.doi:10.3390/mti2040062
71.
SchafheitleS.WeibelA.MeidertN.LeuffenD. (2019). The road to trust. A vignette study on the determinants of citizens’ trust in the European Commission. Journal of Common Market Studies, 1–20.
72.
SchwarzC.GasparJ.BrownT. (2019). The effect of reliability on drivers’ trust and behavior in conditional automation. Cognition, Technology & Work, 21, 41–54.doi:10.1007/s10111-018-0522-y
SiegristM. (in press). Trust and risk perception: A critical review of the literature. Risk Analysis, 27.doi:10.1111/risa.13325
75.
SiegristM.CvetkovichG.RothC. (2000). Salient value similarity, social trust, and risk/benefit perception. Risk Analysis, 20, 353–362.doi:10.1111/0272-4332.203034http://www.ncbi.nlm.nih.gov/pubmed/10949414
76.
SitkinS. B.RothN. L. (1993). Explaining the limited effectiveness of legalistic “remedies” for trust/distrust. Organization Science, 4, 367–392.doi:10.1287/orsc.4.3.367
77.
SonodaK.WadaT. (2017). Displaying system situation awareness increases driver trust in automated driving. IEEE Transactions on Intelligent Vehicles, 2, 185–193.doi:10.1109/TIV.2017.2749178
78.
SponarskiC. C.VaskeJ. J.BathA. J.MusianiM. M. (2014). Salient values, social trust, and attitudes toward wolf management in south-western Alberta, Canada. Environmental Conservation, 41, 303–310.doi:10.1017/S0376892913000593
79.
StockertS.RichardsonN. T.LienkampM. (2015). Driving in an increasingly automated world–approaches to improve the driver-automation interaction. Procedia Manufacturing, 3, 2889–2896.doi:10.1016/j.promfg.2015.07.797
80.
SütfeldL. R.GastR.KönigP.PipaG. (2017). Using virtual reality to assess ethical decisions in road traffic scenarios: Applicability of value-of-life-based models and influences of time pressure. Frontiers in Behavioral Neuroscience, 11, 122.doi:10.3389/fnbeh.2017.00122
81.
SzekelyR. D.MiuA. C. (2015). Incidental emotions in moral dilemmas: The influence of emotion regulation. Cognition and Emotion, 29, 64–75.doi:10.1080/02699931.2014.895300http://www.ncbi.nlm.nih.gov/pubmed/24611625
82.
TomzM. R.WeeksJ. L. P. (2013). Public opinion and the democratic peace. American Political Science Review, 107, 849–865.doi:10.1017/S0003055413000488
83.
VerberneF. M. F.HamJ.MiddenC. J. H. (2012). Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. The Journal of the Human Factors and Ergonomics Society, 54, 799–810.doi:10.1177/0018720812443825http://www.ncbi.nlm.nih.gov/pubmed/23156624
84.
VerberneF. M. F.HamJ.MiddenC. J. H. (2015). Trusting a virtual driver that looks, acts, and thinks like you. The Journal of the Human Factors and Ergonomics Society, 57, 895–909.doi:10.1177/0018720815580749http://www.ncbi.nlm.nih.gov/pubmed/25921302
85.
WaldropM. M. (2015). Autonomous vehicles: No drivers required. Nature, 518, 20–23.doi:10.1038/518020ahttp://www.ncbi.nlm.nih.gov/pubmed/25652978
86.
WalkerG. H.StantonN. A.SalmonP. (2016). Trust in vehicle technology. International Journal of Vehicle Design, 70, 157–182.doi:10.1504/IJVD.2016.074419
87.
XuZ.ZhangK.MinH.WangZ.ZhaoX.LiuP. (2018). What drives people to accept automated vehicles? Findings from a field experiment. Transportation Research Part C: Emerging Technologies, 95, 320–334.doi:10.1016/j.trc.2018.07.024
88.
YamagishiT. (1998). 信 の構 ：こころと社会の進化ゲー [The structure of trust: The evolutionary games of mind and society] (16th ed). University of Tokyo Press.
89.
YokoiR.NakayachiK. (2018). The effects of shared policy of medical treatment on trust in artificial intelligence. Japanese Journal of Social Psychology, 34, 16–25.
90.
YokoiR.NakayachiK. (2019). The effect of shared investing strategy on trust in artificial intelligence. The Japanese Journal of Experimental Social Psychology, 59, 46–50.doi:10.2130/jjesp.1819
91.
ZhangL.LiZ.WuX.ZhangZ. (2017). Why people with more emotion regulation difficulties made a more deontological judgment: The role of deontological inclinations. Frontiers in Psychology, 8, 2095.doi:10.3389/fpsyg.2017.02095http://www.ncbi.nlm.nih.gov/pubmed/29234299
92.
ZhangT.TaoD.QuX.ZhangX.LinR.ZhangW. (2019). The roles of initial trust and perceived risk in public’s acceptance of automated vehicles. Transportation Research Part C: Emerging Technologies, 98, 207–220.doi:10.1016/j.trc.2018.11.018
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
