Effects of In-Vehicle Touchscreen Location on Driver Task Performance,Eye Gaze Behavior,and Workload During Conditionally Automated Driving: Nondriving-Related Task and Take-Over

Abstract

Objective

This study investigated the effects of nondriving-related task (NDRT) touchscreen location and NDRT difficulty level on the driver task performance, eye gaze behavior, and workload during SAE Level 3 conditionally automated driving. Two driver tasks were considered: a visuomanual NDRT and a take-over task.

Background

Touchscreens are expected to play important roles inside automated vehicles. However, few studies have investigated the driver-touchscreen interaction during automated driving.

Method

A driving simulator experiment was conducted. The experimental task consisted of two successive subtasks: an NDRT followed by a take-over task. NDRT touchscreen location (Upper Left, Upper Right, and Lower Right) and NDRT difficulty level (Easy and Hard) were the independent variables. A set of driver task performance, eye gaze behavior, and perceived workload measures were employed for each subtask as the dependent variables.

Results

NDRT touchscreen location significantly affected both the NDRT and the take-over task performance. Lower Right was superior to Upper Right in the NDRT performance but was inferior in the take-over task performance. NDRT touchscreen location affected the perceived physical workload of the NDRT. NDRT difficulty level affected the perceived workload of the take-over task.

Conclusion

The research findings enhance our understanding of how NDRT touchscreen location and NDRT difficulty level impact the driver task performance during conditionally automated driving, and, further provide useful design implications and knowledge.

Application

The study results would inform the NDRT touchscreen interface design and the NDRT design for conditionally automated vehicles.

Keywords

touch screens autonomous driving display design principles visual search task switching

Get full access to this article

View all access options for this article.

References

Ahmad

B. I.

Murphy

J. K.

Langdon

P. M.

Godsill

S. J.

Hardy

Skrypchuk

(2015). Intent inference for hand pointing gesture-based interactions in vehicles. IEEE Transactions on Cybernetics, 46(4), 878–889. https://doi.org/10.1109/TCYB.2015.2417053

Armstead

(2022, August 16). 2022 Ford Mustang Mach-E: Galloping towards the future. Forbes WHEELS. https://www.forbes.com/wheels/cars/ford/mustang-mach-e/

Baker

Olson

Morisseau

(1994). Work practices, fatigue, and nuclear power plant safety performance. Human Factors, 36(2), 244–257. https://doi.org/10.1177/001872089403600206

Bergmann

Deuretzbacher

Heller

Graichen

Rohlmann

Strauss

Duda

G. N.

(2001). Hip contact forces and gait patterns from routine activities. Journal of Biomechanics, 34(7), 859–871. https://doi.org/10.1016/s0021-9290(01)00040-9

Bhise

V. D.

(2011). Ergonomics in the automotive design process. CRC Press.

Burnett

G. E.

Lawson

Donkor

Kuriyagawa

(2013). Menu hierarchies for in- vehicle user-interfaces: Modelling the depth vs. breadth trade-off. Displays, 34(4), 241–249. https://doi.org/10.1016/j.displa.2013.07.001

Chen

Wang

Cheng

Gao

Tremont

P. J.

(2022). Evaluation of the optimal quantity of in-vehicle information icons using a fuzzy synthetic evaluation model in a driving simulator. Accident Analysis & Prevention, 176, 106813. https://doi.org/10.1016/j.aap.2022.106813

Cockburn

Woolley

Thai

K. T. P.

Clucas

Hoermann

Gutwin

(2018, September). Reducing the attentional demands of in-vehicle touchscreens with stencil overlays. Proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications (pp. 33–42). Association for Computing Machinery.

Crundall

Large

D. R.

Burnett

(2016). A driving simulator study to explore the effects of text size on the visual demand of in-vehicle displays. Displays, 43, 23–29. https://doi.org/10.1016/j.displa.2016.05.003

10.

Eren

A. L.

Burnett

Large

D. R.

Harvey

(2018). Understanding the effects of peripheral vision and muscle memory on in-vehicle touchscreen interactions. IET Intelligent Transport Systems, 12(6), 434–439. https://doi.org/10.1049/iet-its.2017.0229

11.

Feng

Liu

Chen

(2018). Effects of quantity and size of buttons of in-vehicle touch screen on drivers’ eye glance behavior. International Journal of Human-Computer Interaction, 34(12), 1105–1118. https://doi.org/10.1080/10447318.2017.1415688

12.

Field

A. P.

Miles

(2009). Discovering statistics using SPSS: (and sex and drugs and rock ‘n’ roll). Sage Publications.

13.

Fuller

Tsimhoni

Reed

M. P.

(2008, September). Effect of in-vehicle touch screen position on driver performance. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 52(23), 1893–1897. https://doi.org/10.1177/154193120805202313

14.

Gold

Damböck

Lorenz

Bengler

(2013). “Take over!” How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 57(1), 1938–1942. https://doi.org/10.1177/1541931213571433

15.

Grahn

Kujala

(2020). Impacts of touch screen size, user interface design, and subtask boundaries on in-car task’s visual demand and driver distraction. International Journal of Human-Computer Studies, 142, 102467. https://doi.org/10.1016/j.ijhcs.2020.102467

16.

Hagiwara

Sakakima

Hashimoto

Kawai

(2013). Effect of distraction on driving performance using touch screen while driving on test track. 2013 IEEE intelligent vehicles symposium (IV) (pp. 1149–1154), Gold Coast, QLD, Australia, 23–26 June 2013.

17.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139–183. https://doi.org/10.1016/s0166-4115(08)62386-9

18.

Hoffman

J. D.

Lee

J. D.

McGehee

D. V.

Macias

Gellatly

A. W.

(2005). Visual sampling of in-vehicle text messages: Effects of number of lines, page presentation, and message control. Transportation Research Record: Journal of the Transportation Research Board, 1937(1), 22–30. https://doi.org/10.1177/0361198105193700104

19.

Huang

McMurran

Amor-Segan

Dhadyalla

Jones

R. P.

Bennett

Mouzakitis

kieloch

(2010). Development of an automated testing system for vehicle infotainment system. The International Journal of Advanced Manufacturing Technology, 51(1–4), 233–246. https://doi.org/10.1007/s00170-010-2626-2

20.

ISO 15007-1 . (2002). Road vehicles–measurement of driver visual behaviour with respect to transport information and control systems–Part 1: Definitions and parameters. ISO International Organization for Standardization, (ISO International Organization for Standardization).

21.

Jancer

(2022, August 30). Review: Rivian R1T and R1S. WIRED. https://www.wired.com/review/rivian-r1t-and-r1s/

22.

Janssen

C. P.

Kun

A. L.

Brewster

Boyle

L. N.

Brumby

D. P.

Chuang

L. L.

(2019, September). Exploring the concept of the (future) mobile office. Proceedings of the 11th international conference on automotive user interfaces and interactive vehicular applications: Adjunct proceedings (pp. 465–467). Association for Computing Machinery.

23.

Karhu

Kansi

Kuorinka

(1977). Correcting working postures in industry: A practical method for analysis. Applied Ergonomics, 8(4), 199–201. https://doi.org/10.1016/0003-6870(77)90164-8

24.

Kato

Endo

Kizuka

(2009). Mental fatigue and impaired response processes: Event-related brain potentials in a Go/NoGo task. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology, 72(2), 204–211. https://doi.org/10.1016/j.ijpsycho.2008.12.008

25.

Kim

Kwon

Heo

Lee

Chung

M. K.

(2014). The effect of touch-key size on the usability of In-Vehicle Information Systems and driving safety during simulated driving. Applied Ergonomics, 45(3), 379–388. https://doi.org/10.1016/j.apergo.2013.05.006

26.

Kujala

Saariluoma

(2011). Effects of menu structure and touch screen scrolling style on the variability of glance durations during in-vehicle visual search tasks. Ergonomics, 54(8), 716–732. https://doi.org/10.1080/00140139.2011.592601

27.

Kujala

Salvucci

D. D.

(2015). Modeling visual sampling on in-car displays: The challenge of predicting safety-critical lapses of control. International Journal of Human-Computer Studies, 79, 66–78. https://doi.org/10.1016/j.ijhcs.2015.02.009

28.

Levin

(2020, October 30). I drove a tesla model Y after testing 14 other electric cars. I loved its sleek interior and easy charging but the giant screen is overkill. INSIDER. https://www.businessinsider.com/tesla-model-y-pros-cons-takeaways-first-drive-2022-10

29.

Liu

Sun

Gao

Zhang

(2022). Exploring the performance of click and slide gestures on large in-vehicle touch screens. Applied Ergonomics, 99, 103613. https://doi.org/10.1016/j.apergo.2021.103613

30.

Ludewig

P. M.

Cook

T. M.

Nawoczenski

D. A.

(1996). Three-dimensional scapular orientation and muscle activity at selected positions of humeral elevation. Journal of Orthopaedic & Sports Physical Therapy, 24(2), 57–65. https://doi.org/10.2519/jospt.1996.24.2.57

31.

Ding

Gong

(2018, December). Impact of screen size and structure of IVIS on driving distraction IOP conference series: Materials science and engineering. IOP Conference Series: Materials Science and Engineering, 452(4), 042154. https://doi.org/10.1088/1757-899x/452/4/042154

32.

Meijman

T. F.

(1997). Mental fatigue and the efficiency of information processing in relation to work times. International Journal of Industrial Ergonomics, 20(1), 31–38. https://doi.org/10.1016/s0169-8141(96)00029-7

33.

Mourant

R. R.

Rockwell

T. H.

(1972). Strategies of visual search by novice and experimental drivers. Human Factors, 14(4), 325–335. https://doi.org/10.1177/001872087201400405

34.

NHTSA

V. M. N. D. D.

(2013). Guidelines for in-vehicle electronic devices. NHTSA-2010-0053. Department of Transportation–National Highway Traffic Safety Administration.

35.

Patel

Liu

Zhao

Liu

(2022, June). Inspection of in-vehicle touchscreen infotainment display for different screen locations, menu types, and positions. International conference on human-computer interaction (pp. 258–279). Springer International Publishing.

36.

Pitts

M. J.

Burnett

Skrypchuk

Wellings

Attridge

Williams

M. A.

(2012b). Visual–haptic feedback interaction in automotive touchscreens. Displays, 33(1), 7–16. https://doi.org/10.1016/j.displa.2011.09.002

37.

Pitts

M. J.

Skrypchuk

Wellings

Attridge

Williams

M. A.

(2012a). Evaluating user response to in-car haptic feedback touchscreens using the lane change test. Advances in Human-Computer Interaction, 2012, 598739. https://doi.org/10.1155/2012/598739

38.

Russomanno

O'Modhrain

Gillespie

(2017, June). A Pneu shape display: Physical buttons with programmable touch response. 2017 IEEE World Haptics Conference (WHC) (pp. 641–646). IEEE.

39.

Samuel

Borowsky

Zilberstein

Fisher

D. L.

(2016). Minimum time to situation awareness in scenarios involving transfer of control from an automated driving suite. Transportation Research Record: Journal of the Transportation Research Board, 2602(1), 115–120. https://doi.org/10.3141/2602-14

40.

Sanders

A. F.

Sanders

(2013). Elements of human performance: Reaction processes and attention in human skill. Psychology Press.

41.

Sanders

M. S.

McCormick

E. J.

(1993). Human factors in engineering and design. McGraw-Hill Companies.

42.

Schölkopf

Wolf

M. M.

Hutmann

Diermeyer

(2021, September). Conception, development and first evaluation of a context-adaptive user interface for commercial vehicles. 13th international conference on automotive user interfaces and interactive vehicular applications (pp. 21–25). Association for Computing Machinery.

43.

Society of Automotive Engineers (SAE) . (2016). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles (Surface Vehicle Recommended Practice: Superseding J3016 Jan 2014). SAE International.

44.

Soomro

Cockburn

(2020, November). Design and evaluation of layout-agnostic tactile guides for in-vehicle touchscreens. 2020 5th international conference on innovative technologies in intelligent systems and industrial applications (CITISIA) (pp. 1–10). IEEE.

45.

Steelman

K. S.

McCarley

J. S.

Wickens

C. D.

(2017). Theory-based models of attention in visual workspaces. International Journal of Human-Computer Interaction, 33(1), 35–43. https://doi.org/10.1080/10447318.2016.1232228

46.

Suh

Ferris

T. K.

(2019). On-road evaluation of in-vehicle interface characteristics and their effects on performance of visual detection on the road and manual entry. Human Factors, 61(1), 105–118. https://doi.org/10.1177/0018720818790841

47.

Tao

Yuan

Liu

(2018). Effects of button design characteristics on performance and perceptions of touchscreen use. International Journal of Industrial Ergonomics, 64, 59–68. https://doi.org/10.1016/j.ergon.2017.12.001

48.

Tullis

Albert

(2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics (2nd ed.). Morgan Kaufmann.

49.

Van der Linden

Frese

Meijman

T. F.

(2003). Mental fatigue and the control of cognitive processes: Effects on perseveration and planning. Acta Psychologica, 113(1), 45–65. https://doi.org/10.1016/s0001-6918(02)00150-6

50.

Wickens

McCarley

Thomas

Foyle

M. I. D.

Goodman

Hooey

B. L.

(2003, September). Attention-situation awareness (A-SA) model. NASA aviation safety program conference on human performance modeling of approach and landing with augmented displays (Vol. 189).

51.

Wickens

C. D.

(2015). Noticing events in the visual workplace: The SEEV and NSEEV models. Cambridge University Press.

52.

Wickens

C. D.

Gordon

S. E.

Liu

Lee

(2004) An introduction to human factors engineering (vol. 2, p. 587). Pearson Prentice Hall.

53.

Wickens

C. D.

Gutzwiller

R. S.

Vieane

Clegg

B. A.

Sebok

Janes

(2016). Time sharing between robotics and process control: Validating a model of attention switching. Human Factors, 58(2), 322–343. https://doi.org/10.1177/0018720815622761

54.

Wickens

C. D.

McCarley

J. S.

(2019). Applied attention theory. CRC Press.

55.

Wickens

C. D.

Santamaria

Sebok

(2013, September). A computational model of task overload management and task switching. Proceedings of the Human Factors and Ergonomics Society - Annual Meeting, 57(1), 763–767. https://doi.org/10.1177/1541931213571167

56.

Wickens

C. D.

Sebok

Sarter

Gacy

A. M.

(2015). Using modeling and simulation to predict operator performance and automation-induced complacency with robotic automation: A case study and empirical validation. Human Factors, 57(6), 959–975. https://doi.org/10.1177/0018720814566454

57.

Wiker

S. F.

Langolf

G. D.

Chaffin

D. B.

(1989). Arm posture and human movement capability. Human Factors, 31(4), 421–441. https://doi.org/10.1177/001872088903100406

58.

Xian

Jin

(2015). The effects of using in-vehicle computer on driver eye movements and driving performance. Advances in Mechanical Engineering, 7(2), 981908. https://doi.org/10.1155/2014/981908