Impacts of Non-Driving-Related Task Interface Design on Time Changes of Motion Sickness Severity and Task Performance During Fully Autonomous Driving

Abstract

Autonomous vehicles (AVs) promise enhanced safety and mobility, allowing passengers to perform non-driving-related tasks (NDRTs). However, motion sickness (MS) poses a significant challenge to AV adoption. This study examined the impact of different NDRT interfaces (Visual vs. Auditory displays, Manual vs. Speech controls) on MS severity and NDRT performance. An experiment with 20 participants was conducted, where each participant completed simulated AV trials for each interface combination. Results showed that Visual displays with Manual control (VM) led to the fastest MS severity increase, while Auditory displays with Speech control (AS) were most effective in mitigating MS. Conversely, VM maintained NDRT performance better over time, while Auditory-Manual (AM) exhibited the steepest performance decline. The findings highlight the need for interface designs that balance MS mitigation and NDRT performance, suggesting further research into adaptive, context-aware AV interfaces to optimize user experience and safety.

Autonomous vehicles (AVs) have the potential to revolutionize transportation by providing benefits such as increased safety and improved mobility for all. They also allow passengers to engage in various non-driving-related tasks (NDRTs), thereby improving productivity and convenience. However, challenges may hinder the widespread adoption of AV technology, one of which is motion sickness (MS).

Carefully choosing interface modalities for NDRT interfaces may help reduce MS and prevent the resulting performance degradation. This is because the design choice can influence how passengers perceive the vehicle’s motion and their own, thereby affecting the amount of sensory conflict, which is known to be the main cause of MS. Hence, the objective of the current study was to compare different NDRT interfaces (Display-Control modality combinations) in terms of the time profiles of MS severity and NDRT performance.

To achieve the research objective, an on-road, within-subject experiment was conducted. Four NDRT interfaces were considered, combining two Display modalities (visual [V] and auditory [A]) and two Control modalities (manual [M] and speech [S]). These combinations were labeled as VM, VS, AM, and AS. Twenty participants took part in the experiment. Each participant completed a single simulated autonomous driving trial for each of the four modality combinations, one trial per day. On each day, they performed the NDRT (a letter 2-back task) using the corresponding modality combination while seated in the assistant seat. Each simulated driving trial lasted 30 min and included 30 letter sequences of 2-back. Participants reported their MS severity using the Fast Motion Sickness (FMS) scale every minute. NDRT performance, measured as the percentage of accurate responses (PAR), was calculated for every minute. Linear mixed regression analysis was conducted on FMS and NDRT PAR data, using Display modality, Control modality, Time, and their interactions as fixed effects.

The results of the linear mixed regression analysis on FMS data demonstrated significant main and interaction effects of Display, Time, Display × Time, Control × Time, and intercept on MS severity. The model for FMS score was: FMS = 1.54−0.47 × Display + 0.08 × Control + 0.30 × Display × Control + 0.17 × Time −0.05 × Display × Time −0.04 × Control × Time+0.01 × Display × Control × Time, where Display = 0 for V and 1 for A, and, Control = 0 for M and 1 for S. The equations for the four Display-Control modality combinations were: VM_FMS = 1.54 + 0.167 × Time, VS_FMS = 1.62 + 0.129 × Time, AM_FMS = 1.08 + 0.116 × Time, and, AS_FMS = 1.41 + 0.090 × Time. The constant term in each equation represents the initial level of MS severity and the slope indicates the rate at which MS severity level changed over Time. The rate of change in MS severity over Time was the largest for VM followed by VS, AM, and AS.

Similarly, NDRT PAR data revealed significant main and interaction effects of Display × Control, Time, Display × Time, Control × Time, Display × Control × Time and intercept on NDRT PAR. The model for NDRT PAR was: NDRT PAR = 98.51 + 0.71 × Display + 1.13 × Control −0.06 × Display × Control −2.15 × Time −12.95 × Display × Time −0.09 × Control × Time + 0.20 × Display × Control × Time. The equations for the four Display-Control modality combinations were: VM_NDRT PAR = 98.51 − 0.06 × Time, VS_NDRT PAR = 99.64−0.15 × Time, AM_NDRT PAR = 99.22−0.19 × Time, and, AS_NDRT PAR = 98.20−0.08 × Time. The rate of change in NDRT PAR over Time was the largest for AM followed by VS, AS, and VM.

Some major study findings were as follows:

• Display, Time, Display × Time and Control × Time significantly affected MS severity over Time. VM led to the fastest increase, while AS was the most effective in mitigating MS.

• Display × Control, Time, Display × Time, Control × Time, Display × Control × Time significantly affected NDRT PAR over Time. VM maintained the best performance throughout the 30-min data collection period, exhibiting the smallest decline in accuracy. AM exhibited the steepest decline in NDRT PAR.

• The results demonstrated that the optimal Display-Control modality combination for mitigating MS severity differed from the combination that best maintained NDRT performance.

Overall, the research findings will inform the design of safer and more user-friendly AVs, contributing to the enhanced adoption and acceptance of AV technology.