Evaluation of Gesture-Based In-Vehicle Interaction: User Experience and the Potential to Reduce Driver Distraction

Abstract

Objective:

We observe the effects of in-vehicle system gesture-based interaction versus touch-based interaction on driver distraction and user experience.

Background:

Driver distraction is a major problem for traffic safety, as it is a contributing factor to a number of accidents. Visual distraction in particular has a highly negative impact on the driver. One possibility for reducing visual driver distraction is to use new forms of interaction in the vehicle, such as gesture-based interaction.

Method:

In this experiment, participants drove on a motorway or in a city scenario while using touch-based interaction or gesture-based interaction. Subjective data, such as acceptance and workload, and objective data, including glance behavior, were gathered.

Results:

As a result, participants rated their subjective impressions of safe driving as higher when using gesture-based interaction. More specifically, acceptance and attractiveness were higher, and workload was lower. The participants performed significantly fewer glances to the display and the glances were much shorter.

Conclusion:

Gestures are a positive alternative for in-vehicle interaction since effects on driver distraction are less significant when compared to touch-based interaction.

Application:

Potential application of this research includes interaction design of typical in-vehicle information and entertainment functions.

Keywords

in-vehicle interaction systems (IVIS)human-machine interaction (HMI)driver distraction

Get full access to this article

View all access options for this article.

References

Alliance of Automobile Manufacturers. (2006). Statement of principles, criteria and verification Procedures on driver interactions with advanced in-vehicle information and communication systems. Washington, DC. Retrieved from http://umich.edu/~driving/publications/documents/DF-T%20with%202006%20-Updates-prot.doc

Bakeman

(2005). Recommended effect size statistics for repeated measures designs. Behavior Research Methods, 37(3), 379–384. https://doi.org/10.3758/BF03192707

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1). https://doi.org/10.18637/jss.v067.i01

Caird

J. K.

Simmons

S. M.

Wiley

Johnston

K. A.

Horrey

W. J.

(2018). Does talking on a cell phone, with a passenger, or dialing affect driving performance? An updated systematic review and meta-analysis of experimental studies. Human Factors: The Journal of the Human Factors and Ergonomics Society, 60(1), 101–133. https://doi.org/10.1177/0018720817748145

Chiesa

Branciforti

. (2012). Experimenting Kinect interactions in the car. In Kun

A. L.

Boyle

L. N.

Reimer

Riener

(Eds.), AutomotiveUI 2012: 4th Internationl Conference on Automotive User Interfaces and Vehicular Applications (Adjunct Proceedings). October 17–19, Portsmouth, NH.

Choudhary

Velaga

N. R.

(2017). Mobile phone use during driving: Effects on speed and effectiveness of driver compensatory behaviour. Accident: Analysis and Prevention, 106, 370–378. https://doi.org/10.1016/j.aap.2017.06.021

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hoboken, NJ: Taylor and Francis. Retrieved from http://gbv.eblib.com/patron/FullRecord.aspx?p=1192162

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555

Dahlbäck

Jönsson

Ahrenberg

(1993). Wizard of Oz studies. In Gray

W. D.

(Ed.), Proceedings of the 1st International Conference on Intelligent User Interfaces (pp. 193–200). New York, NY: ACM. https://doi.org/10.1145/169891.169968

10.

Donges

(2016). Driver behavior models. In Winner

Hakuli

Lotz

Singer

(Eds.), Handbook of driver assistance systems: Basic information, components and systems for active safety and comfort (pp. 19–33). Cham, Germany: Springer International Publishing. https://doi.org/10.1007/978-3-319-12352-3_2

11.

Drews

F. A.

Yazdani

Godfrey

C. N.

Cooper

J. M.

Strayer

D. L.

(2009). Text messaging during simulated driving. Human Factors: The Journal of the Human Factors and Ergonomics Society, 51(5), 762–770. https://doi.org/10.1177/0018720809353319.

12.

Engström

Johansson

Östlund

(2005). Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research Part F: Traffic Psychology and Behaviour, 8(2), 97–120. https://doi.org/10.1016/j.trf.2005.04.012

13.

Engström

Victor

Markkula

(2013). Attention selection and multitasking in everyday driving: A conceptual model. In Regan

M. A.

Lee

J. D.

Victor

T. W.

(Eds.), Driver distraction and inattention. Advances in research and countermeasures. Farnham, UK: Ashgate.

14.

Fariman

H. J.

Alyamani

H. J.

Kavakli

Hamey

(2016). Designing a user-defined gesture vocabulary for an in-vehicle climate control system. In Duh

Lueg

Billinghust

Huang

(Eds.), Proceedings of the 28th Australian Computer-Human Interaction Conference (OzCHI 2016): 29 November - 2 December 2016, University of Tasmania (pp. 391–395). New York, NY: ACM. https://doi.org/10.1145/3010915.3010955

15.

Feng

Marulanda

Donmez

(2014). Susceptibility to driver distraction questionnaire. Transportation Research Record: Journal of the Transportation Research Board, 2434, 26–34. https://doi.org/10.3141/2434-04

16.

Geiger

Nieschulz

Zobl

Lang

(2002). Bedienkonzept zur gestenbasierten Interaktion mit Geräten im Automobil (Gesture-based control concept for in-car devices). In VDI-Berichte 1678, GMA Fachtagung USEWARE 2002. Darmstadt, Germany, June 11–12. Düsseldorf: VDI Verlag.

17.

Graichen

Krems

J. F.

(2017a). Do users want additional feedback when using a gesture-based system? In Blashki

(Ed.), Proceedings of the International Conference Interfaces and Human Computer Interaction 2017 (Part of the Multi Conference on Computer Science and Information Systems (MCCSIS) (pp. 93–99). Lisbon, Portugal, July 21–23, 2017. IADIS Press.

18.

Graichen

Krems

J. F.

(2017b). Gestures in human-computer interaction: Which shape and how many? In Blashki

(Ed.), Proceedings of the International Conference Interfaces and Human Computer Interaction 2017 (part of the Multi Conference on Computer Science and Information Systems (MCCSIS) (pp. 69–76). Lisbon, Portugal, July 21–23, 2017. IADIS Press.

19.

Greenberg

Tijerina

Curry

Artz

Cathey

Kochhar

. . . Grant

(2003). Driver distraction: Evaluation with event detection paradigm. Transportation Research Record: Journal of the Transportation Research Board, 1843, 1–9. https://doi.org/10.3141/1843-01

20.

Hancock

P. A.

(2013). Driven to distraction and back again. In Regan

M. A.

Lee

J. D.

Victor

T. W.

(Eds.), Driver distraction and inattention. Advances in research and countermeasures. Farnham, UK: Ashgate.

21.

Hanley

P. F.

Sikka

(2012). Bias caused by self-reporting distraction and its impact on crash estimates. Accident Analysis & Prevention, 49, 360–365. https://doi.org/10.1016/j.aap.2012.02.008

22.

Hart

S. G.

(2006). Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 50(9), 904–908. https://doi.org/10.1177/154193120605000909

23.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in Psychology. Human Mental Workload (Vol. 52, pp. 139–183). Amsterdam, the Netherlands: Elsevier. https://doi.org/10.1016/S0166-4115(08)62386-9

24.

Hassenzahl

Burmester

Koller

. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität [AttrakDiff: A questionnaire for measuring perceived hedonic and pragmatic quality]. In Szwillus

Ziegler

(Eds.), Berichte des German Chapter of the ACM: Vol. 57. Mensch & Computer 2003: Interaktion in Bewegung (pp. 187–196). Stuttgart, Germany: Vieweg+Teubner Verlag.

25.

Hassenzahl

Burmester

Koller

(2008). Der User Experience (UX) auf der Spur: Zum Einsatz von www.attrakdiff.de. In Brau

Diefenbach

Hassenzahl

Koller

Peissner

Röse

(Eds.), Usability Professionals 2008: Berichtband des sechsten Workshops des German Chapters der Usability Professionals Association e.V. 7. bis 10. September 2008, Lübeck (pp. 78–82). Stuttgart: German Chapter der Usability Professionals Assoc.

26.

Horrey

W. J.

Lesch

M. F.

(2009). Driver-initiated distractions: Examining strategic adaptation for in-vehicle task initiation. Accident Analysis & Prevention, 41(1), 115–122. https://doi.org/10.1016/j.aap.2008.10.008

27.

Horrey

W. J.

Wickens

C. D.

(2007). In-vehicle glance duration: Distributions, tails, and model of crash risk. Transportation Research Record: Journal of the Transportation Research Board, 2018(1), 22–28. https://doi.org/10.3141/2018-04

28.

Jian

J.-Y.

Bisantz

A. M.

Drury

C. G.

(2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53–71. https://doi.org/10.1207/S15327566IJCE0401_04

29.

Kaber

D. B.

Liang

Zhang

Rogers

M. L.

Gangakhedkar

(2012). Driver performance effects of simultaneous visual and cognitive distraction and adaptation behavior. Transportation Research Part F: Traffic Psychology and Behaviour, 15(5), 491–501. https://doi.org/10.1016/j.trf.2012.05.004

30.

Kircher

Ahlstrom

(2017). Minimum required attention: A human-centered approach to driver inattention. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(3), 471–484. https://doi.org/10.1177/0018720816672756

31.

Kuznetsova

Brockhoff

P. B.

Christensen

R. H. B.

(2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13). https://doi.org/10.18637/jss.v082.i13

32.

Laird

N. M.

Ware

J. H.

(1982). Random-effects models for longitudinal data. Biometrics, 38(4), 963. https://doi.org/10.2307/2529876

33.

Lansdown

T. C.

(2001). Causes, measures, and effects of driver visual workload. In Hancock

P. A.

Desmond

P. A.

(Eds.), Human factors in transportation. Stress, workload, and fatigue. Mahwah, NJ: Erlbaum.

34.

Lausberg

Sloetjes

(2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, 41(3), 841–849. https://doi.org/10.3758/brm.41.3.841

35.

Lawrence

M. A.

(2015). ez: Easy analysis and visualization of factorial experiments. Retrieved from https://CRAN.R-project.org/package=ez

36.

Lee

J. D.

Roberts

S. C.

Hoffman

J. D.

Angell

L. S.

(2012). Scrolling and driving: How an MP3 player and its aftermarket controller affect driving performance and visual behavior. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(2), 250–263. https://doi.org/10.1177/0018720811429562

37.

Lee

J. D.

Young

K. L.

Regan

M. A.

(2009). Defining driver distraction. In Regan

M. A.

Lee

J. D.

Young

K. L.

(Eds.), Driver distraction: Theory, effects, and mitigation. Boca Raton, FL: CRC Press.

38.

Liang

Lee

J. D.

(2010). Combining cognitive and visual distraction: Less than the sum of its parts. Accident Analysis & Prevention, 42(3), 881–890. https://doi.org/10.1016/j.aap.2009.05.001

39.

März

Schwahlen

Geisler

Kopinski

. User expectations on touchless gestures in vehicles. In Weyers

Dittmar

(Eds.), Mensch und computer 2016: Workshopband. Aachen: Gesellschaft für Informatik e.V. https://doi.org/10.18420/MUC2016-WS08-0002

40.

May

Gable

T. M.

Sardesai

R. R.

Walker

B. N.

(2016). Choosing the right air gesture: Impacts of menu length and air gesture type on driver workload. In Green

Pfleging

Kun

A. L.

Liang

Meschtscherjakov

Fröhlich

(Eds.), AutomotiveUI 2016: 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Adjunct proceedings) (pp. 69–74). Ann Arbor, MI, USA, October 24–26, 2016. https://doi.org/10.1145/3004323.3004330

41.

McGehee

D. V.

(2001). New design guidelines aim to reduce driver distractiontion. Human Factors and Ergonomics Society Bulletin, 44(10), 1–3.

42.

Metz

Landau

Just

(2014). Frequency of secondary tasks in driving: Results from naturalistic driving data. Safety Science, 68, 195–203. https://doi.org/10.1016/j.ssci.2014.04.002

43.

National Center for Statistics and Analysis. (2017). Distracted driving 2015. (Traffic Safety Facts Research Note. Report No. DOT HS 812 381). Washington, DC: National Highway Traffic Safety Administration. Retrieved from https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812381

44.

National Highway Traffic Safety Administration. (2010). Overview of the national highway traffic safety administration’s driver distraction program. (DOT HS 811 299). Washington, DC: US Department of Transportation.

45.

Peng

Boyle

L. N.

(2015). Driver’s adaptive glance behavior to in-vehicle information systems. Accident Analysis & Prevention, 85, 93–101. https://doi.org/10.1016/j.aap.2015.08.002

46.

Regan

M. A.

Hallett

Gordon

C. P.

(2011). Driver distraction and driver inattention: Definition, relationship and taxonomy. Accident Analysis & Prevention, 43(5), 1771–1781. https://doi.org/10.1016/j.aap.2011.04.008

47.

M. A.

Regan

J. D.

Lee

Victor

T. W.

(Eds.). (2013). Driver distraction and inattention. Advances in research and countermeasures. Farnham, UK: Ashgate.

48.

M. A.

Regan

J. D.

Lee

Young

K. L.

(Eds.). (2009). Driver distraction. Theory, effects, and mitigation. Boca Raton, FL: CRC Press.

49.

Reissner

(2007). Gestures and speech in cars. In Matiyasevich

Mayr

E. W.

(Eds.), Electronic proceedings of Joint Advanced Student School (JASS). March 25–April 4, St. Petersburg, Russia.

50.

SAE International. (2017). Definition and experimental measures related to the specification of driver visual behavior using video based techniques (J-2396). Warrendale, PA: Society of Automotive Engineers, Inc. Retrieved from https://www.sae.org/standards/content/j2396_200007/

51.

Saffer

(2008). Designing gestural interfaces. Sebastopol, CA: O’Reilly Media, Inc.

52.

Saifuzzaman

Haque

M. M.

Zheng

Washington

(2015). Impact of mobile phone use on car-following behaviour of young drivers. Accident Analysis & Prevention, 82, 10–19. https://doi.org/10.1016/j.aap.2015.05.001

53.

Sawyer

B. D.

Finomore

V. S.

Calvo

A. A.

Hancock

P. A.

(2014). Google Glass. Human Factors: The Journal of the Human Factors and Ergonomics Society, 56(7), 1307–1321. https://doi.org/10.1177/0018720814555723

54.

Schaap

Van der Horst

van Arem

Brookhuis

(2013). The relationship between driver distraction and mental workload. In Regan

M. A.

Lee

J. D.

Victor

T. W.

(Eds.), Driver distraction and inattention: Advances in research and countermeasures. Farnham, UK: Ashgate.

55.

Senders

J. W.

(2013). Driver distraction and inattention: A queuing theory approach. In Regan

M. A.

Lee

J. D.

Victor

T. W.

(Eds.), Driver distraction and inattention. Advances in research and countermeasures. Farnham, UK: Ashgate.

56.

Tippey

K. G.

Sivaraj

Ferris

T. K.

(2017). Driving while interacting with Google Glass: Investigating the combined effect of head-up display and hands-free input on driving safety and multitask performance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(4), 671–688. https://doi.org/10.1177/0018720817691406

57.

Tivesten

Dozza

(2015). Driving context influences drivers’ decision to engage in visual-manual phone tasks: Evidence from a naturalistic driving study. Journal of Safety Research, 53, 87–96. https://doi.org/10.1016/j.jsr.2015.03.010

58.

Van Buuren

Groothuis-Oudshoorn

. (2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. Retrieved from http://www.jstatsoft.org/v45/i03/

59.

Van der Laan

J. D.

Heino

Waard

D. de

. (1997). A simple procedure for the assessment of acceptance of advanced transport telematics. Transportation Research Part C: Emerging Technologies, 5(1), 1–10. https://doi.org/10.1016/S0968-090X(96)00025-3

60.

Van Laack

Kirsch

Tuzar

G.-D.

Blessing

. (2016). Consumer insights about gesture interaction in vehicles. Visteon White Paper. Retrieved from https://www.visteon.com/products/technical-publications.html

61.

Vollrath

Huemer

A. K.

Teller

Likhacheva

Fricke

(2016). Do German drivers use their smartphones safely? Not really! Accident; analysis and prevention, 96, 29–38. https://doi.org/10.1016/j.aap.2016.06.003

62.

Wierwille

W. W.

(1993). Visual and manual demands of in-car controls and displays. In Peacock

Karwowski

(Eds.), Automotive Ergonomics. London, UK: Taylor & Francis.

63.

Young

K. L.

Salmon

P. M.

Cornelissen

(2013). Distraction-induced driving error: An on-road examination of the errors made by distracted and undistracted drivers. Accident Analysis & Prevention, 58, 218–225. https://doi.org/10.1016/j.aap.2012.06.001

64.

Zhang

Angell

(2014). Pointing towards future automotive HMIs. In Boyle

L. N.

Kun

A. L.

Pearce

Szostak

Osswald

(Eds.), AutomotiveUI ‘14: 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Adjunct Proceedings). September 17–19, 2014, Seattle, WA. New York, NY: ACM. https://doi.org/10.1145/2667239.2667297