This study examines the extent to which cybersecurity attacks on autonomous vehicles (AVs) affect human trust dynamics and driver behavior.
Background
Human trust is critical for the adoption and continued use of AVs. A pressing concern in this context is the persistent threat of cyberattacks, which pose a formidable threat to the secure operations of AVs and consequently, human trust.
Method
A driving simulator experiment was conducted with 40 participants who were randomly assigned to one of two groups: (1) Experience and Feedback and (2) Experience-Only. All participants experienced three drives: Baseline, Attack, and Post-Attack Drive. The Attack Drive prevented participants from properly operating the vehicle in multiple incidences. Only the “Experience and Feedback” group received a security update in the Post-Attack drive, which was related to the mitigation of the vehicle’s vulnerability. Trust and foot positions were recorded for each drive.
Results
Findings suggest that attacks on AVs significantly degrade human trust, and remains degraded even after an error-less drive. Providing an update about the mitigation of the vulnerability did not significantly affect trust repair.
Conclusion
Trust toward AVs should be analyzed as an emergent and dynamic construct that requires autonomous systems capable of calibrating trust after malicious attacks through appropriate experience and interaction design.
Application
The results of this study can be applied when building driver and situation-adaptive AI systems within AVs.
AhmadU.SongH.BilalA.AlazabM.JolfaeiA. (2020). Securing smart vehicles from relay attacks using machine learning. The Journal of Supercomputing, 76(4), 2665–2682. https://doi.org/10.1007/s11227-019-03049-4
2.
AhnD.AlmaatouqA.GulabaniM.HosanagarK. (2021). Will we trust what we don’t understand? Impact of model interpretability and outcome feedback on trust in AI. arXiv preprint arXiv:2111.08222.
3.
AkhtarN.MianA. (2018). Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access, 6, 14410–14430. https://doi.org/10.1109/access.2018.2807385
4.
BlancoM.AtwoodJ.VasquezH.TrimbleT.FitchettV.RadlbeckJ.FitchG.RussellS.GreenC.CullinaneB.MorganJ. (2015). Human factors evaluation of level 2 and level 3 automated driving concepts. In Technical report. NHTSA.
5.
ChenM.NikolaidisS.SohH.HsuD.SrinivasaS. (2020). Trust-aware decision making for human-robot collaboration: Model learning and planning. ACM Transactions on Human-Robot Interaction (THRI), 9(2), 1–23. https://doi.org/10.1145/3359616
6.
CugurulloF.AcheampongR. A. (2023). Fear of AI: An inquiry into the adoption of autonomous cars in spite of fear, and a theoretical framework for the study of artificial intelligence technology acceptance. AI & SOCIETY, 39(4), 1569–1584. https://doi.org/10.1007/s00146-022-01598-6
7.
CummingsM. L.KilgoreR. M.WangE.TijerinaL.KochharD. S. (2007). Effects of single versus multiple warnings on driver performance. Human Factors, 49(6), 1097–1106. https://doi.org/10.1518/001872007X249956
8.
DesaiM.KaniarasuP.MedvedevM.SteinfeldA.YancoH. (2013). Impact of robot failures and feedback on real-time trust. In 2013 8th ACM/IEEE international conference on human-robot interaction (HRI) (pp. 251–258). IEEE.
9.
De VisserE. J.PakR.ShawT. H. (2018). From ‘automation’ to ‘autonomy’: The importance of trust repair in human–machine interaction. Ergonomics, 61(10), 1409–1427. https://doi.org/10.1080/00140139.2018.1457725
10.
DikmenM.BurnsC. (2017). Trust in autonomous vehicles: The case of Tesla Autopilot and summon. In 2017 IEEE international conference on systems, man, and cybernetics (SMC) (pp. 1093–1098). IEEE.
DzindoletM. T.PetersonS. A.PomrankyR. A.PierceL. G.BeckH. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58(6), 697–718. https://doi.org/10.1016/s1071-5819(03)00038-7
13.
EykholtK.EvtimovI.FernandesE.LiB.RahmatiA.XiaoC.PrakashA.KohnoT.SongD. (2018). Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1625–1634). IEEE.
14.
GouribhatlaR.PulugurthaS. S. (2022). Drivers’ behavior when driving vehicles with or without advanced driver assistance systems: A driver simulator-based study. Transportation Research Interdisciplinary Perspectives, 13(15), Article 100545. https://doi.org/10.1016/j.trip.2022.100545
15.
GrossmanR.FeitosaJ. (2018). Team trust over time: Modeling reciprocal and contextual influences in action teams. Human Resource Management Review, 28(4), 395–410. https://doi.org/10.1016/j.hrmr.2017.03.006
HolthausenB. E.WintersbergerP.WalkerB. N.RienerA. (2020). Situational trust scale for automated driving (STS-AD): Development and initial validation. In 12th international conference on automotive user interfaces and interactive vehicular applications (pp. 40–47). ACM.
18.
JianJ.-Y.BisantzA. M.DruryC. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71. https://doi.org/10.1207/s15327566ijce0401_04
19.
KhanS. K.ShiwakotiN.StasinopoulosP. (2022). A conceptual system dynamics model for cybersecurity assessment of connected and autonomous vehicles. Accident Analysis & Prevention, 165(2), 106515. https://doi.org/10.1016/j.aap.2021.106515
20.
KohnS. C.QuinnD.PakR.De VisserE. J.ShawT. H. (2018). Trust repair strategies with self-driving vehicles: An exploratory study. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 62(1), 1108–1112. https://doi.org/10.1177/1541931218621254
21.
KoxE. S.KerstholtJ. H.HuetingT. F.de VriesP. W. (2021). Trust repair in human-agent teams: The effectiveness of explanations and expressing regret. Autonomous Agents and Multi-Agent Systems, 35(2), 30. https://doi.org/10.1007/s10458-021-09515-9
22.
KrausJ.ScholzD.StiegemeierD.BaumannM. (2020). The more you know: Trust dynamics and calibration in highly automated driving and the effects of take-overs, system malfunction, and system transparency. Human Factors, 62(5), 718–736. https://doi.org/10.1177/0018720819853686
23.
KundingerT.WintersbergerP.RienerA. (2019). (Over)Trust in automated driving: The sleeping pill of tomorrow? In Extended abstracts of the 2019 CHI conference on human factors in computing systems (Glasgow, Scotland UK) (CHI EA ’19). Association for Computing Machinery.
24.
LeeJ.MorayN. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243–1270. https://doi.org/10.1080/00140139208967392
LiN.BoyleL. N. (2020). Allocation of driver attention for varying in-vehicle system modalities. Human Factors, 62(8), 1349–1364. https://doi.org/10.1177/0018720819879585
27.
LimC.RajivanP. (2023). Who hacked my car? Designing autonomous vehicles to support driver response to security threats. In Proceedings of the human factors and ergonomics society annual meeting (p. 21695067231192217). Sage Publications Sage CA.
28.
LiuJ.MarriottK.DwyerT.TackG. (2023). Increasing user trust in optimisation through feedback and interaction. ACM Transactions on Computer-Human Interaction, 29(5), 1–34. https://doi.org/10.1145/3503461
29.
LodgeD. (2016). Hacking the Mitsubishi Outlander PHEV hybrid. PenTestPartners, June 5.
30.
MackayA.FortesI.SantosC.MachadoD.BarbosaP.Vilas BoasV.FerreiraJ. P.CostaN.SilvaC.SousaE. (2020). The impact of autonomous vehicles’ active feedback on trust. In Advances in safety management and human factors: Proceedings of the AHFE 2019 international conference on safety management and human factors, 24–28 July, 2019, Washington DC, USA (pp. 342–352). Springer.
31.
MadhavanP.WiegmannD. A. (2007). Similarities and differences between human–human and human–automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8(4), 277–301. https://doi.org/10.1080/14639220500337708
32.
MalikS.SunW. (2020). Analysis and simulation of cyber attacks against connected and autonomous vehicles. In 2020 international conference on connected and autonomous driving (MetroCAD) (pp. 62–70). IEEE.
33.
McNeeseN.DemirM.ChiouE.CookeN.YanikianG. (2019). Understanding the role of trust in human-autonomy teaming.
34.
McNeeseN. J.DemirM.ChiouE. K.CookeN. J. (2021). Trust and team performance in human–autonomy teaming. International Journal of Electronic Commerce, 25(1), 51–72. https://doi.org/10.1080/10864415.2021.1846854
35.
Meyer-WaardenL.CloarecJ. (2022). “Baby, you can drive my car”: Psychological antecedents that drive consumers’ adoption of AI-powered autonomous vehicles. Technovation, 109(2), Article 102348. https://doi.org/10.1016/j.technovation.2021.102348
36.
MillerC.ValasekC. (2015). Remote exploitation of an unaltered passenger vehicle. In Black Hat USA 2015, S, 91 (pp. 1–91).
37.
MorayN.InagakiT. (1999). Laboratory studies of trust between humans and machines in automated systems. Transactions of the Institute of Measurement and Control, 21(4-5), 203–211. https://doi.org/10.1177/014233129902100408
38.
MorraL.LambertiF.PratticóF. G.RosaS. L.MontuschiP. (2019). Building trust in autonomous vehicles: Role of virtual reality driving simulators in HMI design. IEEE Transactions on Vehicular Technology, 68(10), 9438–9450. https://doi.org/10.1109/tvt.2019.2933601
39.
NielsenJ. (2024). Usability heuristics for user interface design.
40.
PayreW.CestacJ.DangN.-T.VienneF.DelhommeP. (2017). Impact of training and in-vehicle task performance on manual control recovery in an automated car. Transportation Research Part F: Traffic Psychology and Behaviour, 46, 216–227. https://doi.org/10.1016/j.trf.2017.02.001
41.
PetitJ.ShladoverS. E. (2015). Potential cyberattacks on automated vehicles. IEEE Transactions on Intelligent Transportation Systems, 16(2), 546–556. https://doi.org/10.1109/TITS.2014.2342271
42.
PhamM.XiongK. (2021). A survey on security attacks and defense techniques for connected and autonomous vehicles. Computers & Security, 109(12), Article 102269. https://doi.org/10.1016/j.cose.2021.102269
43.
SatoT.HayakawaY.SuzukiR.ShiikiY.YoshiokaK.ChenQ. A. (2024, February). LiDAR spoofing meets the new-gen: Capability improvements, broken assumptions, and new attack strategies. In ISOC network and distributed system security symposium (NDSS). ISOC.
44.
SchneiderT.HoisJ.RosensteinA.GhellalS.Theofanou-FülbierD.GerlicherA. R. S. (2021). Explain yourself! Transparency for positive ux in autonomous driving. In Proceedings of the 2021 CHI conference on human factors in computing systems. CHI.
45.
ShariffA.BonnefonJ.-F.RahwanI. (2017). Psychological roadblocks to the adoption of self-driving vehicles. Nature Human Behaviour, 1(10), 694–696. https://doi.org/10.1038/s41562-017-0202-6
46.
ShenJ.WonJ. Y.ChenZ.ChenQ. A. (2020). Drift with devil: Security of {Multi-Sensor} fusion based localization in {High-Level} autonomous driving under {GPS} spoofing. In 29th USENIX security symposium (USENIX Security 20) (pp. 931–948). USENIX.
47.
SheridanT. B. (2012). Human supervisory control. In Handbook of human factors and ergonomics (pp. 990–1015). John Wiley & Sons.
48.
ThingV. L. L.WuJ. (2016). Autonomous vehicle security: A taxonomy of attacks and defences. In 2016 IEEE international conference on internet of things (iThings) and IEEE green computing and communications (GreenCom) and IEEE cyber, physical and social computing (CPSCom) and IEEE smart data (SmartData) (pp. 164–170). IEEE.
WrightT. J.HorreyW. J.LeschM. F.RahmanM. M. (2016). Drivers’ trust in an autonomous system: Exploring a covert video-based measure of trust. In Proceedings of the human factors and ergonomics society annual meeting (Vol. 60, pp. 1334–1338). Sage Publications Sage CA.
51.
WulffD. U.Mergenthaler-CansecoM.HertwigR. (2018). A meta-analytic review of two modes of learning and the description-experience gap. Psychological Bulletin, 144(2), 140–176. https://doi.org/10.1037/bul0000115
52.
YuB.BaoS.ZhangY.SullivanJ.FlannaganM. (2021). Measurement and prediction of driver trust in automated vehicle technologies: An application of hand position transition probability matrix. Transportation Research Part C: Emerging Technologies, 124(5), Article 102957. https://doi.org/10.1016/j.trc.2020.102957
