We observe the driving performance effects of gesture-based interaction (GBI) versus touch-based interaction (TBI) for in-vehicle information systems (IVISs).
Background
As a contributing factor to a number of traffic accidents, driver distraction is a significant problem for traffic safety. More specifically, visual distraction has a strong negative impact on driving performance and risk perception. Thus, the implementation of new interaction systems that use midair gestures to encourage glance-free interactions could reduce visual distraction among drivers.
Methods
In this experiment, participants drove a projection-based Vehicle-in-the-Loop. The projection-based technology combines a visual simulation with kinesthetic, vestibular, and auditory feedback from a car on a test track. While driving, participants used GBI or TBI to perform IVIS tasks. To investigate driving behavior related to critical driving situations and car-following maneuvers, vehicle data based upon longitudinal and lateral driving were collected.
Results
Participants reacted faster to critical driving situations when using GBI compared to TBI. For drivers using TBI, steering performance decreased and time headway to a preceding vehicle was higher.
Conclusion
Gestures provide a safe alternative to in-vehicle interactions. Moreover, GBI has fewer effects on driver distraction than TBI.
Application
Potential applications of this research include all in-vehicle interaction systems used by drivers.
AhmadB.LangdonP. M.GodsillS. J.DonkorR.WildeR.SkrypchukL. (2016). You do not have to touch to select: A study on predictive in-car touchscreen with mid-air selection. InGreenP.BollS.BurnettG.GabbardJ.OsswaldS. (Eds.), Icps, AutomotiveUI 2016: 8th International conference on automotive user interfaces and interactive vehicular applications (proceedings). October 24–26, 2016, Ann Arbor, MI, USA (pp.113–120). Association for Computing Machinery. https://doi.org/10.1145/3003715.3005461.
2.
AngeliniL.CarrinoF.CarrinoS.CaonM.KhaledO. A.BaumgartnerJ.MugelliniE. (2014). Gesturing on the steering wheel: A user-elicited taxonomy. InBoyleL. N.FröhlichP.BurnettG.IqbalS. (Eds.), AutomotiveUI 2014: 6th International conference on automotive user interfaces and interactive vehicular applications (proceedings). September 17-19, 2014, Seattle, Washington, USA (pp.1–8). Association for Computing Machinery. https://doi.org/10.1145/2667317.2667414.
3.
BatesD.MächlerM.BolkerB.WalkerS. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67.doi:10.18637/jss.v067.i01
4.
BergG.NitschV.FärberB. (2016). Vehicle in the loop. InWinnerH.HakuliS.LotzF.SingerC. (Eds.), Handbook of driver assistance systems: Basic information, components and systems for active safety and comfort (pp.199–210). Springer International Publishing. https://doi.org/10.1007/978-3-319-12352-3_10.
5.
BockT. (2012). Bewertung von Fahrerassistenzsystemen mittels der Vehicle in the loop-simulation. InWinnerH.HakuliS.WolfG. (Eds.), ATZ/MTZ-Fachbuch. Handbuch Fahrerassistenzsysteme. Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort (2nd ed.,pp.76–83). Vieweg + Teubner. https://doi.org/10.1007/978-3-8348-8619-4_9.
BurnettG.CrundallE.LargeD.LawsonG.SkrypchukL. (2013). A study of unidirectional swipe gestures on in-vehicle touch screens. InTerkenJ. (Ed.), Proceedings of the 5th International conference on automotive user interfaces and interactive vehicular applications, Eindhoven, 2013, [October 28 - 30, 2013] (pp.22–29). ACM. https://doi.org/10.1145/2516540.2516545.
8.
CairdJ. K.SimmonsS. M.WileyK.JohnstonK. A.HorreyW. J. (2018). Does talking on a cell phone, with a passenger, or Dialing affect driving performance? An updated systematic review and meta-analysis of experimental studies. Human Factors: The Journal of the Human Factors and Ergonomics Society, 60, 101–133.doi:10.1177/0018720817748145
9.
ChiesaM.BrancifortiA. (2012). Experimenting Kinect interactions in the car. InKunA. L.BoyleL. N.ReimerB.RienerA. (Eds.), AutomotiveUI 2012: 4th international conference on automotive user interfaces and vehicular applications (Adjunct Proceedings). October 17-19, 2012. Portsmouth, New Hampshire, US.
10.
ChoudharyP.VelagaN. R. (2017). Mobile phone use during driving: Effects on speed and effectiveness of driver compensatory behaviour. Accident Analysis & Prevention, 106, 370–378.doi:10.1016/j.aap.2017.06.02128715728
11.
DahlbäckN.JönssonA.AhrenbergL. (1993). Wizard of Oz studies. InGrayW. D. (Ed.), Proceedings of the 1st international conference on intelligent user interfaces (pp.193–200). ACM. https://doi.org/10.1145/169891.169968.
12.
DongesE. (2016). Driver behavior models. InWinnerH.HakuliS.LotzF.SingerC. (Eds.), Handbook of driver assistance systems: Basic information, components and systems for active safety and comfort (pp.19–33). Springer International Publishing. https://doi.org/10.1007/978-3-319-12352-3_2.
13.
DöringT.KernD.MarshallP.PfeifferM.SchöningJ.GruhnV.SchmidtA. (2011). Gestural interaction on the steering wheel. InTanD.FitzpatrickG.GutwinC.BegoleB.KelloggW. A. (Eds.), Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11 (p.483). ACM Press. https://doi.org/10.1145/1978942.1979010.
14.
DrewsF. A.YazdaniH.GodfreyC. N.CooperJ. M.StrayerD. L. (2009). Text messaging during simulated driving. Human Factors: The Journal of the Human Factors and Ergonomics Society, 51, 762–770.doi:10.1177/001872080935331920196300
15.
EckerR.BroyV.HertzschuchK.ButzA. (2010). Visual cues supporting direct touch gesture interaction with in-vehicle systems. InBollS.DeyA. K.KunA. L.SchmidtA. (Eds.), 2nd International conference on automotive user intefaces and interactive vehicular applications (AutomotiveUI 2010). November 11-12, 2010. Pittsburgh, Pennsylvania, USA. Carnegie Mellon University.
16.
EngströmJ.JohanssonE.ÖstlundJ. (2005). Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research Part F: Traffic Psychology and Behaviour, 8, 97–120.doi:10.1016/j.trf.2005.04.012
17.
FangV.AinsworthL. (2012). Designing & rapid prototyping a gesture-enabled steering wheel. InKunA. L.BoyleL. N.ReimerB.RienerA. (Eds.), AutomotiveUI 2012: 4th International conference on automotive user interfaces and vehicular applications (Adjunct Proceedings). October 17-19, 2012. Portsmouth, New Hampshire, US.
18.
FarimanH. J.AlyamaniH. J.KavakliM.HameyL. (2016). Designing a user-defined gesture vocabulary for an in-vehicle climate control system. InDuhH.LuegC.BillinghustM.HuangW. (Eds.), Proceedings of the 28th Australian computer-human interaction conference (OzCHI 2016). 29 November - 2 December 2016, University of Tasmania (pp.391–395). ACM. https://doi.org/10.1145/3010915.3010955.
19.
FieldA.MilesJ.FieldZ. (2012). Discovering statistics using R (Reprint). Sage.
20.
GeigerM.NieschulzR.ZoblM.LangM. (2002). Bedienkonzept zur gestenbasierten Interaktion mit Geräten im Automobil (Gesture-based control concept for in-car devices)[Conference session]. VDI-Berichte 1678, GMA Fachtagung USEWARE 2002. Darmstadt, Germany, June 11-12. VDI Verlag
21.
GeigerM.ZoblM.BenglerK.LangM. (2001). Intermodal differences in distraction effects while controlling automotive user interfaces. InSmithM. J.SalvendyG.HarrisD.KoubekR. J. (Eds.), Human factors and ergonomics: Jointly with symposium on human interface (Japan) 2001, 4th international conference on engineering psychology and cognitive ergonomics, 1st international conference on universal access in human-computer interaction; August 5 - 10, 2001 in New Orleans, Louisiana, USA; Vol. 1. Usability evaluation and interface design: Cognitive engineering, intelligent agents and virtual reality (pp.263–267). Lawrence Erlbaum.
22.
GraichenL.GraichenM.KremsJ. F. (2017a). Do users want additional feedback when using a gesture-based system?InBlashkiK. (Ed.), Proceedings of the international conference interfaces and human computer interaction 2017 (part of the multi conference on computer science and information systems (MCCSIS). Lisbon, Portugal, July 21 - 23, 2017 (pp.93–99). IADIS Press.
23.
GraichenL.GraichenM.KremsJ. F. (2017b). Gestures in Human-Computer Interaction - Which shape and how many?InBlashkiK. (Ed.), Proceedings of the international conference interfaces and human computer interactiOn 2017 (part of the multi conference on computer science and information systems (MCCSIS). Lisbon, Portugal, July 21 - 23, 2017 (pp.69–76). IADIS Press.
24.
GraichenL.GraichenM.KremsJ. F. (2019). Evaluation of gesture-based in-vehicle interaction: User experience and the potential to reduce driver distraction. Human Factors: The Journal of the Human Factors and Ergonomics Society, 61, 774–792.doi:10.1177/001872081882425330694705
25.
GraichenM.GraichenL.RottmannT.NitschV. (2018). Using the projection-based Vehicle in the Loop for the investigation of in-vehicle information systems: First insights. InHelfertM.GusikhinO. (Eds.), VEHITS 2018: Proceedings of the 4th international conference on vehicle technology and intelligent transport systems. Funchal, Portugal, March 16-18, 2018. SciTePress.
26.
GreenbergJ.TijerinaL.CurryR.ArtzB.CatheyL.KochharD.GrantP. (2003). Driver distraction: Evaluation with event detection paradigm. Transportation research record. Journal of the Transportation Research Board, 1843, 1–9.
27.
HorreyW. J.LeschM. F. (2009). Driver-initiated distractions: Examining strategic adaptation for in-vehicle task initiation. Accident Analysis & Prevention, 41, 115–122.doi:10.1016/j.aap.2008.10.00819114145
28.
KaberD. B.LiangY.ZhangY.RogersM. L.GangakhedkarS. (2012). Driver performance effects of simultaneous visual and cognitive distraction and adaptation behavior. Transportation Research Part F: Traffic Psychology and Behaviour, 15, 491–501.doi:10.1016/j.trf.2012.05.004
29.
KarlI.BergG.RügerF.FärberB. (2013). Driving behavior and simulator sickness while driving the vehicle in the loop: Validation of longitudinal driving behavior. IEEE Intelligent Transportation Systems Magazine, 5, 42–57.doi:10.1109/MITS.2012.2217995
30.
KopinskiT.EberweinJ.GeislerS.HandmannU. (2016). Touch versus mid-air gesture interfaces in road scenarios - Measuring driver performance degradation. In2016 IEEE 19th international conference on intelligent transportation systems (pp.661–666). IEEE.
31.
KuznetsovaA.BrockhoffP. B.ChristensenR. H. B. (2017). lmerTest Package: Tests in linear mixed effects models. Journal of Statistical Software, 82.doi:10.18637/jss.v082.i13
32.
LairdN. M.WareJ. H. (1982). Random-effects models for longitudinal data. Biometrics, 38, 963.doi:10.2307/25298767168798
33.
LeeJ. D.RobertsS. C.HoffmanJ. D.AngellL. S. (2012). Scrolling and driving: How an MP3 player and its aftermarket controller affect driving performance and visual behavior. Human Factors, 54, 250–263.doi:10.1177/001872081142956222624291
34.
LeeJ. D.YoungK. L.ReganM. A. (2009). Defining driver distraction. InReganM. A.LeeJ. D.YoungK. L. (Eds.), Driver distraction: Theory, effects, and mitigation. CRC Press.
35.
LeeS. H.YoonS.-O.ShinJ. H. (2015). On-wheel finger gesture control for in-vehicle systems on central consoles. InBurnettG.GabbardJ.GreenP.OsswaldS.KunA.PflegingB.AntrobusV. (Eds.), AutomotiveUI '15: 7th International conference on automotive user interfaces and interactive vehicular applications (Adjunct Proceedings) (pp.94–99). Association for Computing Machinery. https://doi.org/10.1145/2809730.2809739.
36.
LiangY.LeeJ. D. (2010). Combining cognitive and visual distraction: Less than the sum of its parts. Accident Analysis & Prevention, 42, 881–890.doi:10.1016/j.aap.2009.05.00120380916
37.
MahrA.EndresC.SchneebergerT.MüllerC. (2011). Determining human-centered parameters of ergonomic micro-gesture interaction for drivers using the theater approach. InTscheligiM.KranzM.WeinbergG.MeschtscherjakovA.MurerM.WilfingerD. (Eds.), Automotive UI '11: 3rd international conference on automotive user interfaces and interactive vehicular applications (Proceedings). November 30th - December 2nd, 2011, Salzburg, Austria. Association for Computing Machinery.
38.
MairP.WilcoxR. (2018). WRS2: Wilcox robust estimation and testing.
39.
MärzP.SchwahlenD.GeislerS.KopinskiT. (2016). User Expectations on Touchless Gestures in Vehicles. InWeyersB.DittmarA. (Eds.), Mensch und Computer 2016 - Workshopband. Gesellschaft für Informatik e.V. https://doi.org/10.18420/MUC2016-WS08-0002.
40.
MayK.GableT. M.WuX.SardesaiR. R.WalkerB. N. (2016). Choosing the right air gesture: Impacts of menu length and air gesture type on driver workload. InGreenP.PflegingB.KunA. L.LiangY.MeschtscherjakovA.FröhlichP. (Eds.), AutomotiveUI 2016: 8th international conference on automotive user interfaces and interactive vehicular applications (Adjunct proceedings). Ann Arbor, MI, USA, October 24-26, 2016.
41.
MayK. R.GableT. M.WalkerB. N. (2017). Designing an in-vehicle air gesture set using elicitation methods. InBollS.PflegingB.DonmezB. (Eds.), AutomotiveUI 2017: The 9th international conference on automotive user interfaces and interactive vehicular applications (Proceedings). Oldenburg, Germany, September 24-27 (pp. 74–83). Association for Computing Machinery. https://doi.org/10.1145/3122986.3123015.
42.
MetzB.LandauA.JustM. (2014). Frequency of secondary tasks in driving – results from naturalistic driving data. Safety Science, 68, 195–203.doi:10.1016/j.ssci.2014.04.002
Neumann-CoselKvon.DupuisM.WeissC. (2009). Virtual test drive provision of a consistent toolset for [D,H,S,V]-in-the-loop[Conference session]. Proceedings of DSC 2009 Europe Driving Simulation Conference. Monaco, Monaco, January 4-6, 2009.
45.
NHTSA. (2010). Overview of the national highway traffic safety administration's driver distraction program (DOT HS 811 299). US Department of Transportation.
46.
ÖstlundJ.PetersB.ThorslundJ.KeinathD.HorstS.FoehlU.MattesS.EngströmG.MarkkulaA. (2005). Driving performance assessment methods and metrics. (Deliverable 2.2.5 of the AIDE project No. IST-1-507674-IP, SP2). http://www.aide-eu.org/pdf/sp2_deliv_new/aide_d2_2_5.pdf.
47.
PengY.BoyleL. N. (2015). Driver's adaptive glance behavior to in-vehicle information systems. Accident Analysis & Prevention, 85, 93–101.doi:10.1016/j.aap.2015.08.00226406538
48.
ReganM. A.HallettC.GordonC. P. (2011). Driver distraction and driver inattention: Definition, relationship and taxonomy. Accident Analysis & Prevention, 43, 1771–1781.doi:10.1016/j.aap.2011.04.00821658505
49.
ReganM. A.LeeJ. D.VictorT. W.. (Eds.). (2013). Driver distraction and inattention: Advances in research and countermeasures. Ashgate.
ReissnerU. (2007). Gestures and speech in cars. InMatiyasevichY.MayrE. W. (Eds.), Electronic proceedings of Joint Advanced Student School (JASS). March 25 - April 4, 2007. Russia.
52.
RiedlB.FärberB. (2015). Evaluation of a new projection concept for the Vehicle in the Loop (VIL) driving simulator. InBülthoffH.KemenyA.PrettoP. (Eds.), Proceedings of DSC 2015 Europe driving simulation conference & exhibition: Tübingen, Germany, September 16 - 18, 2015. Symposium conducted at the meeting of Max-Planck-Institut für Biologische Kybernetik; Proceedings. DSC Europe Driving Simulation Conference & Exhibition; DSC Europe. Driving Simulation Conference and Exhibition.
53.
RienerA.WegerF.FerschaA.BachmairF.HagmüllerP.LemmeA.TappeA. (2013). Standardization of the in-car gesture interaction space. InTerkenJ. (Ed.), Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications, Eindhoven, 2013, [October 28 - 30, 2013] (pp.14–21). ACM. https://doi.org/10.1145/2516540.2516544.
54.
SafferD. (2008). Designing gestural interfaces. O’Reilly Media, Inc.
55.
SaifuzzamanM.HaqueM. M.ZhengZ.WashingtonS. (2015). Impact of mobile phone use on car-following behaviour of young drivers. Accident Analysis & Prevention, 82, 10–19.doi:10.1016/j.aap.2015.05.00126009990
56.
SawyerB. D.FinomoreV. S.CalvoA. A.HancockP. A. (2014). Google glass: A driver distraction cause or cure?Human Factors: The Journal of the Human Factors and Ergonomics Society, 56, 1307–1321.
57.
SchaapN.Van der HorstR.van AremB.BrookhuisK. (2013). The relationship between driver distraction and mental workload. InReganM. A.LeeJ. D.VictorT. W. (Eds.), Driver distraction and inattention: Advances in research and countermeasures. Ashgate.
58.
SendersJ. W. (2013). Driver distraction and inattention - A queuing theory approach. InReganM. A.LeeJ. D.VictorT. W. (Eds.), Driver distraction and inattention: Advances in research and countermeasures. Ashgate.
59.
ShakeriG.WilliamsonJ. H.BrewsterS. (2017). Novel multimodal feedback techniques for in-car mid-air gesture interaction. InBollS.PflegingB.DonmezB. (Eds.), AutomotiveUI 2017: The 9th international conference on automotive user interfaces and interactive vehicular applications (Proceedings). Oldenburg, Germany, September 24-27, (pp.84–93). Association for Computing Machinery. https://doi.org/10.1145/3122986.3123011.
60.
SterkenburgJ.LandryS.JeonM. (2017). Eyes-free in-vehicle gesture controls. InBollS.LöckenA.SchroeterR.BaumannM.AlvarezI.ChuangL.OsswaldS. (Eds.), AutomotiveUI 2017: The 9th international conference on automotive user interfaces and interactive vehicular applications (Adjunct Proceedings). Oldenburg, Germany, September 24-27, 2017 (pp.195–200). Association for Computing Machinery. https://doi.org/10.1145/3131726.3131747.
61.
TheeuwesJ.AlferdinckJ. W. A. M.PerelM. (2002). Relation between glare and driving performance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 44, 95–107.doi:10.1518/001872002449477512118876
62.
TippeyK. G.SivarajE.FerrisT. K. (2017). Driving while interacting with Google glass: Investigating the combined effect of head-up display and hands-free input on driving safety and Multitask performance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59, 671–688.doi:10.1177/001872081769140628186420
63.
TivestenE.DozzaM. (2015). Driving context influences drivers' decision to engage in visual-manual phone tasks: Evidence from a naturalistic driving study. Journal of Safety Research, 53, 87–96.doi:10.1016/j.jsr.2015.03.01025934001
64.
Transportation Research Board. (2010). HCM 2010. Highway capacity manual: Volume 1: Concepts. Transportation Research Board.
65.
VollrathM.HuemerA. K.TellerC.LikhachevaA.FrickeJ. (2016). Do German drivers use their smartphones safely?-Not really!Accident Analysis & Prevention, 96, 29–38.doi:10.1016/j.aap.2016.06.00327497054
66.
WeberS.BlumK.ErnstbergerA.FärberB. (2015). Standard reactions – driver reactions in critical driving situations. Procedia Manufacturing, 3, 2489–2496.doi:10.1016/j.promfg.2015.07.510
67.
WernerS. (2014). The steering wheel as a touch interface. InBoyleL. N.FröhlichP.BurnettG.IqbalS. (Eds.), AutomotiveUI 2014: 6th international conference on automotive user interfaces and interactive vehicular applications (Proceedings). September 17-19, 2014, Seattle, Washington, USA (pp.1–4). Association for Computing Machinery. https://doi.org/10.1145/2667239.2667299.
68.
WigdorD.WixonD.. (Eds.). (2011). Brave NUI world: Designing natural user interfaces for touch and gesture. Safari Tech Books Online: Morgan Kaufmann/Elsevier.
69.
WilcoxR. (2012). Introduction to robust estimation and hypothesis testing (3rd ed.). Elsevier.
70.
YoungK. L.SalmonP. M.CornelissenM. (2013). Distraction-induced driving error: An on-road examination of the errors made by distracted and undistracted drivers. Accident Analysis & Prevention, 58, 218–225.doi:10.1016/j.aap.2012.06.00122727884
71.
ZhangY.AngellL. (2014). Pointing Towards Future Automotive HMIs. InBoyleL. N.KunA. L.PearceB.SzostakD.OsswaldS. (Eds.), AutomotiveUI '14: 6th international conference on automotive user interfaces and interactive vehicular applications (Adjunct Proceedings). September 17-19, 2014, Seattle, WA, USA. ACM. https://doi.org/10.1145/2667239.2667297.
72.
ZieglerJ.DöringR.PfefferJ.UrbasL. (2014). Hand gesture recognition as means for mobile human computer interaction in adverse working environments. InBlashkiK.IsaiasP. (Eds.), Premier reference source: Emerging research and trends in interactivity and the human-computer interface (pp.331–352). Information Science Reference/IGI Global. https://doi.org/10.4018/978-1-4666-4623-0.ch017.
