Highly automated driving systems are expected to require the design of new user-vehicle interactions. Sonification can be used to provide contextualized alarms and cues that can increase situation awareness and user experience. In this study, we examined user perceptions of potential use cases for level 4 automated vehicles in online focus group interviews (N=12). Also, in a driving simulator study, we evaluated (1) visual-only display; (2) non-speech with visual display; and (3) speech with visual display with 20 young drivers. Results indicated participants’ interest in the use cases and insight on desired functions in highly automated vehicles. Both audiovisual display conditions resulted in higher situation awareness for drivers than the visual-only condition. Some differences were found between the non-speech and speech conditions suggesting benefits of sonification for both driving and non-driving related auditory use cases. This study will provide guidance on sonification design for highly automated vehicles.
Get full access to this article
View all access options for this article.
References
1.
BlattnerM. MSumikawaD. A.GreenbergR. M. (1989). Earcons and icons: Their structure and common design principles. Human-Computer Interaction, 4(1), 11–44.
2.
ChoiJ. K.JiY. G. (2015). Investigating the importance of trust on adopting an autonomous vehicle. International Journal of Human-Computer Interaction, 37(10), 692–702.
3.
DingierT.LindsayJ.WalkerB. N. (2008). Learnabiltiy of sound cues for environmental features: Auditory icons, earcons, spear cons, and speech. Proceedings of the International Community for Auditory Display.
4.
EndsleyM. R. (1988). Situation awareness global assessment technique (SAGAT). Proceedings of the IEEE 1988 national aerospace and electronics conference.
5.
FakhrhosseiniS. M.LandryS.TanY. Y.BhattaraiS.JeonM. (2014). If you’re angry, turn the music on: Music can mitigate anger effects on driving performance. AutomotiveUI 2014 - 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, in Cooperation with ACM SIGCHI - Proceedings(September). doi:10.1145/2667317.2667410
6.
GableT. M.WalkerB. N. (2013). Georgia Tech Simulator Sickness Screening Protocol. Retrieved from
7.
GangN.SibiS.MichonR.MokB.ChafeC.JuW. (2018). Don t Be A larmed: Sonifying A utonomous Vehicle Perception to Increase Situation Awareness. Paper presented at the Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications.
8.
GaverW. W. (1986). Auditory icons: Using sound in computer interfaces. Human-Computer Interaction, 2(2), 167177.
9.
HancockP. A.KajaksT.CairdJ. K.ChignellM. H.MizobuchiS.BurnsP. C.… NoyI. Y. (2020). Challenges to human drivers in increasingly automated vehicles. Human Factors, 62(2), 310–328.
10.
HaspielJ.DuN.MeyersonJ.RobertL. P.TilburyD.YangX. J.PradhanA. K. (2018). Explanations and Expectations: Trust Building in Automated Vehicles. ACM/IEEE International Conference on Human-Robot Interaction(Dc), 119–120. doi:10.1145/3173386.3177057
11.
HockP.KrausJ.WalchM.LangN.BaumannM. (2017). Elaborating Feedback Strategies for Maintaining Automation in Highly Automated Driving. 105–112. doi:10.1145/3003715.3005414
12.
JeonM. (2019). Multimodal Displays for Take-over in Level 3 Automated Vehicles while Playing a Game. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–6).
13.
JianJ.-Y.BisantzA. M.DruryC. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71.
14.
LandryS.JeonM.FakhrHosseiniM.TascarellaD. (2016). Listen to Your Drive : An In-Vehicle Sonification Prototyping Tool for Driver State and Performance Data. Adjunct Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ‘16), 21–26. doi:10.1109/CIMCA.2005.1631387
15.
LeeS. C.KoS.SanghaviH.JeonM. (2019). Autonomous driving with an agent: Speech style and embodiment. Adjunct Proceedings - 11th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2019(September), 209–214. doi:10.1145/3349263.3351515
16.
LeeS. C.NadriC.SanghaviH.JeonM. (2020). Exploring User Needs and Design Requirements in Fully Automated Vehicles. Paper presented at the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems.
17.
LitmanT. (2020). Autonomous vehicle implementation predictions: Implications for transport planning.
18.
LiuY. C. (2001). Comparative study of the effects of auditory, visual and multimodality displays on drivers’ performance in advanced traveler information systems. Ergonomics, 44(4), 425–442. doi:10.1080/00140130010011369
19.
MauneyB. S.WalkerB. N. (2004). Creating functional and livable soundscapes for peripheral monitoring of dynamic data. Proceedings of the International Community for Auditory Display.
20.
NeesM. A.WalkerB. N. (2011). Auditory Displays for In-Vehicle Technologies. Reviews of Human Factors and Ergonomics, 7(1), 58–99. doi:10.1177/1557234X11410396
21.
PetermeijerS.BazilinskyyP.BenglerK.de WinterJ. (2017). Take-over again: Investigating multimodal and directional TORs to get the driver back into the loop. Applied Ergonomics, 62, 204–215. doi:10.1016/j.apergo.2017.02.023
22.
SabicE.HenningD.MacDonaldJ. (2019). Adaptive Auditory Alerts for Smart In-Vehicle Interfaces. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1), 1545–1549. doi:10.1177/1071181319631404
23.
SAE Committee. (2014). Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems. SAE Standard J3016, 1–16.
24.
SchoopE.SmithJ.HartmannB. (2018). HindSight: Enhancing Spatial Awareness by Sonifying Detected Objects in Real-Time 360-Degree Video. Paper presented at the Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems.
25.
von SawitzkyT.WintersbergerP.RienerA.GabbardJ. L. (2019). Increasing trust in fully automated driving: route indication on an augmented reality head-up display. Proceedings of the 8th ACM International Symposium on Pervasive Displays.
26.
WalkerB. N.LindsayJ.NanceA.NakanoY.PalladinoD. K.DinglerT.JeonM. (2013). Spearcons (speech-based earcons) improve navigation performance in advanced auditory menus. Human Factors, 55(1), 157182. doi:10.1177/0018720812450587
