Effects of Automation Exposure and Non-Driving Related Tasks on Manual Driving Performance after Takeover of Manual Control

Abstract

Prolonged exposure to an automated driving system (ADS) can lead drivers to acclimate to the ADS in maladaptive ways. For example, exposure to a conditional ADS that maintained a time headway of 0.3 s resulted in drivers adopting smaller time headways postautomation, which is attributed to behavioral adaptation. The current study investigated whether drivers adapt to an ADS’ performance over time and if this adaptation differs based on the duration of automation exposure (15 or 25 min) and the task performed during the automation (NDRT or driving-related task). A greater exposure to the automation did not lead drivers to adopt smaller time headways. Surprisingly, drivers who provided a higher evaluation of the automation adopted smaller time headways post-automation than drivers who provided a lower evaluation. Future research should explore the relationship between trust in ADS and driving behavior to guide the development of safer ADSs.

Introduction

Automotive manufacturers are focused on the development and refinement of automated driving systems (a collection of automated controls; ADS; NHSTA, n.d.) which can assist drivers by shifting responsibility of some driving tasks (e.g., lane-keeping assistance) to the automation to aid drivers (Parasuraman & Riley, 1997). However, although an ADS can aid drivers, it also changes the dynamic of the driving task. Parasuraman and Riley (1997) proposed that an automation system generally adds a new interactional component between operators and the automation. A key issue in the successful implementation of autonomous driving is the degree to which drivers remain in the loop (i.e., the control loop in which elements of a system are operated; Merat et al., 2019). Many ADS partially remove drivers from being in control of the vehicle (i.e., on-the-loop) and others completely remove driver control (i.e., out-of-the-loop; Merat et al., 2019). Current ADS advise drivers to always pay attention to the road and be prepared to take over control (e.g., General Motor Company, n.d.). However, when the driver relinquishes vital aspects of active vehicle control, drivers (a) can divert their attention toward other engaging tasks (Yamani & Horrey, 2018) and (b) may rely on the automation for a longer period of time.

These behaviors during the reliance on driving automation may have negative consequences for drivers’ manual driving performance. For example, due to behavioral adaptation effects, a prolonged exposure to the ADS’ roadway performance (e.g., the following distance it maintains to lead vehicles) may change drivers’ manual driving performance after they regain control. Specifically, human operators may “adapt” their manual driving performance by adopting a similar driving performance that the automation exhibited because they have seen the automation do it and, thus, perceive it as safe (OECD Research Group, 1990; Skottke et al., 2014). Brandenburg and Skottke’s (2014) results supported this: After exposure to a conditional ADS, drivers adopted smaller headway distances post-automation compared to their pre-automation headway distances for multiple 1 km intervals after regaining control of the vehicle. Skottke et al. (2014) suggested that this was caused by the drivers being exposed to a new frame-of-reference of safe and effective driving from the automation.

For behavioral adaptation effects to occur, drivers putatively must monitor the automation’s performance (e.g., the automation maintains a time headway of .3 s; Dragutinovic et al., 2005; OECD Research Group, 1990); thus, the effects of behavioral adaptation may depend on the duration of exposure to the automation (Brandenburg & Skottke, 2014). However, Brandenburg and Skottke (2014) did not include an automation duration variable to compare the effects of behavioral adaptation for drivers exposed to different durations (e.g., shorter duration compared to a longer duration). Additionally, Brandenburg and Skottke (2014) did not consider behaviors associated with the interaction with the ADS (e.g., drivers attending to secondary non-driving related tasks; NDRTs). Engaging in a secondary task can shift drivers’ attention away from the roadway environment and places drivers out-of-the-loop (Merat et al., 2019) which can lead to drivers experiencing a skill decay performance decrement (a degraded performance due to the non-usage of skills; Kaber & Endsley, 1997; Parasuraman et al., 2000) in their manual driving performance. Given that automated vehicle technologies are improving, it is likely that ADS may operate safely for increasingly longer durations in the near future. Thus, it is imperative to investigate the performance differences in (1) adaptation effects between drivers exposed to shorter and longer automation durations and (2) driving performance decrements from adaptation effects and skill decay.

The current study is a follow-up to Brandenburg and Skottke (2014). We measured effects of the task performed during the automation (an NDRT or a driving-related task), and the duration of exposure to high-level driving automation on driving performance, after a driver regains manual control of a vehicle. We expected that longer exposure to an ADS would result in greater decreases in manual driving performance over time (e.g., smaller time headway, greater variability in lateral lane positioning) than shorter exposures. We also expected an interaction between the duration of automation exposure and the task drivers performed during the automation, such that drivers exposed to a longer duration of automation driving while engaged in an NDRT would exhibit a greater manual driving performance decrement (e.g., greater variability in lateral lane positioning) than drivers exposed to a shorter duration of automated driving while engaged in a driving-related task.

Method

Subjects

Twenty-four undergraduate students from Rice University who were licensed drivers and had normal/corrected-to-normal vision, and did not have hearing or motor deficiencies, participated in exchange for course credit. This research complied with the American Psychological Association Code of Ethics and was approved by the Institutional Review Board at Rice University. Informed consent was obtained from each participant.

Procedure

Twelve participants were assigned to a 15-min automation duration condition and the other 12 were assigned to a 25-min automation duration condition. Half of all participants were assigned to a NDRT condition (read a National Geographic magazine) and the other half were assigned to a driving-related task condition (monitor the roadway environment). Participants utilized a STISIM driving simulator for the entire experiment; the automation was implemented via STISIM’s autonomous driving framework (Eriksson et al., 2018) to specify specific automation behavior (e.g., the automation maintains a time headway of .3 s to a lead vehicle). Participants completed a 4-min practice drive to familiarize themselves with the driving simulator and to learn how to activate and deactivate the automation when the simulator prompted them with visual-auditory cues. After completing the practice drive, participants began the experiment.

Drivers drove manually for 10 min while the simulator collected driving performance metrics (mean time headway and the standard deviation of lateral lane positioning; SDLP). The last minute of this pre-automation drive represented baseline performance. Drivers then activated the automation when the simulator prompted them to activate the automation and drivers performed their assigned task (reading a National Geographic magazine or monitoring automation) while the automation was active (15 or 25 min depending on assigned condition). After the total automation duration had passed, the simulator prompted drivers to regain control of the vehicle and drivers drove manually for 10 min while the simulator collected data in ten 1-min intervals (0–1, 1–2, . . ., 9–10). Drivers then completed a post-study questionnaire which contained a 5-item Likert scale asking them to “rate their evaluation of the automation” (1—Poor, 5—Good) and to complete a free response section to provide reasons for their rating. Lastly, each dependent variable (mean time headway and SDLP) was converted into difference scores (post-automation − pre-automation) for each post-automation interval for our analysis.

Results

Mean Time Headway

A 2 (Automation Duration: 15 min, 25 min) × 2 (Task: NDRT, driving-related task) × 10 (Interval: 0–1 min, 1–2 min, . . ., 9–10 min) mixed ANOVA was used to analyze mean time headway, with automation duration and task as between-subject factors and interval as a within-subject factor. There was no significant main effects of task, F(1, 20) = 1.30, p = .27, η _p ² = .06, automation duration, F(1, 20) = .04, p = .85, η_p² < .01, or interval, F(9, 180) = .97, p = .41, η_p² = .05. Additionally, there was no significant interaction between automation duration and task, F(1, 20) = .004, p = .95, η_p² < .01, between task and interval, F(9, 180) = .38, p = .75, η _p ² = .02, or between automation duration and interval, F(9, 180) = .85, p = .46, η _p ² = .041. The three-way interaction between automation duration, task, and interval also was not significant, F(9, 180) = .5, p = .66, η _p ² = .03.

SDLP

A 2 (Automation Duration) × 2 (Task) × 10 (Interval) ANOVA for SDLP indicated a significant main effect of automation duration, F(1, 20) = 19.550, p < .001, η _p ² = .263, but no main effects of task, F(1, 20) = .15, p = .71, η_p² < .01, or interval, F(9, 180) = 1.79, p = .07, η _p ² = .05. There was a significant interaction between automation duration and interval, F(9, 180) = 2.04, p = .04, η _p ² = .06, but not between automation duration and task, F(1, 20) = 2.20, p = .15, η _p ² = .04, or between task and interval, F(9, 180) = 1.07, p = .39, η _p ² = .03. The three-way interaction between task, automation duration, and interval was not significant, F(9, 180) = .78, p = .64, η _p ² = .02.

The two-way interaction between automation duration and interval was followed up with one-way ANOVAs. There were significant main effects of automation duration at the 1st, 2nd, 3rd, 5th, 6th, 7th, and 10th intervals, p < .05. Drivers in the 15-min condition had significantly larger SDLPs post-automation than drivers in the 25-min condition for these intervals.

Discussion

Our results did not support our expectation of behavioral adaptation effects on drivers’ manual driving performance. The longer exposure to the automation did not lead to drivers adopting smaller time headways as hypothesized. We considered several reasons for these results. First, our statistical power may have been insufficient to detect effects on time headway. Although we used G*Power’s a priori power analysis to obtain a sample size estimate, we estimated the effect size to be .25. An effect size of .25 for mean time headway may have been insufficient to obtain the effects we were interested in.

Second, drivers’ evaluation of the automation’s performance may have been a confounding factor for adaptation to occur. Indeed, some of our participants who provided low evaluation ratings (rating of 2 or less) verbally reported that they felt that the automation “would crash” (e.g., “Followed almost perfectly, but was unnecessarily close to the front vehicle to feel safe. I thought it would crash,” “It didn’t crash, but any immediate decrease in the lead vehicle’s speed would lead to an immediate crash”). Additionally, participants who provided high evaluation ratings (rating of 4 or more) gave positive remarks about the automation (e.g., “It was very good at maintaining distance,” “It stayed within the lane and turned smoothly”).

A comparison in time headway differences (post-automation - pre-automation) between drivers who provided a higher evaluation of the automation (rating of 4 or above) and drivers who provided a lower evaluation (rating of 2 or below) revealed differences in time headway based on the evaluation rating. Drivers who provided higher evaluations adopted smaller time headways post-automation (see Figure 1). In other words, drivers who provided a high evaluation of the automation’s performance reduced their time headway post-automation whereas drivers who provided a poor evaluation had similar or larger time headways. This supports our interpretation given that drivers who provided a lower evaluation rating of the automation did not adopt reduced time headways (i.e., adaptation did not occur).

Figure 1.

Time headway differences (post − pre) by interval for groups of evaluation ratings.

Drivers’ evaluation rating of the automation’s performance may be a proxy variable for trust in the automation. Drivers’ comments explaining the reasoning behind their provided evaluation rating of the automation’s performance seem to describe an aspect of trust: predictability (defined as “the degree to which future behavior can be anticipated”; Körber et al., 2018; Lee & See, 2004). Indeed, their comments suggest that they could not fully anticipate (i.e., predict) that the automation would safely avoid crashes because of the automation’s small following distance. This may be reflected by our two groups of drivers who provided higher and lower evaluation ratings. For example, drivers who provided higher ratings may have had a better predictability of the automation whereas drivers who provided lower ratings may have had a lower predictability of the automation. This interpretation would be supported by our finding that there are differences in adopted time headways post-automation between drivers who provided higher evaluations (rating of 4 or more) and drivers who provided lower evaluations (rating of 2 or less; see Figure 1). The implication is that perceived safety and trust in the automation moderated behavioral adaptation. A limitation of the current study is that trust was not measured. Future research should explore the relationship between trust in ADS and behavioral adaptation effects.

Moreover, our finding that drivers in the shorter automation duration condition had larger SDLPs than drivers in the greater automation duration condition is contrary to what would be expected from out-of-the-loop performance decrements (Kaber & Endsley, 1997; Parasuraman et al., 2000). The mechanism that putatively deteriorates SDLP performance is skill decay due to the non-usage of skills (Kaber & Endsley, 1997; Parasuraman et al., 2000). Accordingly, drivers in the 25-min condition should have had larger SDLPs than drivers in the 15-min condition. The lack of this result may be attributed to potential anticipatory effects because drivers in the 25-min automation condition had the takeover request prompted to them closer to the end of the study duration than drivers in the 15-min automation condition (assuming they were able to monitory the passage of time effectively).

The study took about an hour to complete, and participants were aware that the study would last that long (in exchange for equivalent course credit); knowledge of a task’s duration is a significant component of time perception (Tobin & Grondin, 2012). However, drivers in the 25-min automation condition were prompted to resume control of the vehicle closer to the study’s maximum duration (e.g., the takeover request was prompted at the 42-min marker) than drivers in the 15-min automation condition (e.g., the takeover request was prompted at the 32-min marker). Additionally, drivers were aware that the simulator would prompt them to resume control of the vehicle because they practiced the maneuver during the practice trial. Because drivers in the 25-min automation condition encountered the takeover request closer to the maximum study duration in comparison to drivers in the 15-min condition, drivers in the 25-min condition may have been better able to anticipate when the takeover request initiated (assuming they were able to monitor the passage of time effectively). We assume that this is the case given that the secondary tasks are monotonous; when drivers are bored, they may seek out other engaging tasks (Carsten et al., 2012). Thus, because drivers in the 25-min automation condition were exposed to the monotonous task for a longer duration, they may have actively sought out a more engaging task (e.g., regaining control of the vehicle) to avoid the monotony.

When drivers are aware of an impending takeover request, they can execute it without experiencing a performance decrement (Samani et al., 2022); this might explain why drivers in the 25-min condition have a lower SDLP than drivers in the 15-min condition. However, to anticipate when the takeover request would occur, drivers would have needed to attend to how much time has passed. Tobin and Grondin (2012) mentions that for accurate human time perception and time tracking, attention to how much time has passed and knowledge of the task duration (i.e., duration of the study) is needed. Although participants had access to the latter piece of information, we cannot confirm if participants attended to how much time had passed or if they were able to monitor the passage of time effectively. It is possible that drivers may have shifted their attention from the secondary task during the automation to other tasks to avoid the monotonous nature of the task (Carsten et al., 2012). Future work should investigate anticipatory effects on takeover performance when comparing two groups of varying automation durations and drivers’ ability to estimate how much time has passed when operating an ADS.

These results have implications for traffic safety. If drivers adopt a larger time headway due to their distrust in the ADS, it could reduce traffic efficiency and curb the market penetration of ADS. Drivers who adopt a larger time headway after deactivating an automated driving mode could exhibit unpredictable road behavior in mixed-traffic environments (traffic conditions that are comprised of both conventional and automated vehicles). Guidelines for traffic safety emphasize that a safe traffic environment is a predictable one and that drivers should drive “predictably” to prevent injury and death (Temecula, n.d.). Predictability is important because traffic environments require drivers to make inferences and decisions (e.g., whether a specific following distance is safe, if nearby drivers are a threat, and the intention of other drivers; “will they let me overtake them?”). Unpredictable traffic can lead to incorrect inferences and decisions and thus crashes that result in injury or death (Temecula, n.d.). With this in mind, if a driver of an automated vehicle that maintained a small time headway (.3 s) in a mixed-traffic environment were to regain control and adopt a larger time headway (3 s), this would be unpredictable to other road users in the traffic environment. The practical implication is that ADS manufacturers should design their automated driving mode to adhere to safe following distances to mitigate drivers’ adoption of smaller time headways after taking over vehicle control.