Sage Journals: Discover world-class research

Abstract

Objectives

This study aimed to investigate drivers’ disengagement from nondriving related tasks (NDRT) during scheduled takeovers and to evaluate its impact on takeover performance.

Background

During scheduled takeovers, drivers typically have sufficient time to prepare. However, inadequate disengagement from NDRTs can introduce safety risks.

Method

Participants experienced scheduled takeovers using a driving simulator, undergoing two conditions, with and without an NDRT. We assessed their takeover performance and monitored their NDRT disengagement from visual, cognitive, and physical perspectives.

Results

The study examined three NDRT disengagement timings (DTs): DT1 (disengaged before the takeover request), DT2 (disengaged after the request but before taking over), and DT3 (not disengaged). The impact of NDRT on takeover performance varied depending on DTs. Specifically, DT1 demonstrated no adverse effects; DT2 impaired takeover time, while DT3 impaired both takeover time and quality. Additionally, participants who displayed DT1 exhibited longer eye-off-NDRT duration and a higher eye-off-NDRT count during the prewarning stage compared to those with DT2 and DT3.

Conclusion

Drivers can benefit from earlier disengagement from NDRTs, demonstrating resilience to the adverse effects of NDRTs on takeover performance. The disengagement of cognition is often delayed compared to that of eyes and hands, potentially leading to DT3. Moreover, visual disengagement from NDRTs during the prewarning stage could distinguish DT1 from the other two.

Application

Our study emphasizes considering NDRT disengagement in designing systems for scheduled takeovers. Measures should be taken to promote early disengagement, facilitate cognitive disengagement, and employ visual disengagement during the prewarning period as predictive indicators of DTs.

Keywords

distractions and interruptions task switching highly automated vehicles freeway exiting eye-tracking stimulated recall

Introduction

Automated vehicles are categorized into six levels, spanning from Level 0 to Level 5. Level 3 (L3) and Level 4 (L4) Automated Driving Systems (ADSs) can execute driving tasks within their Operational Designed Domain (ODD) (SAE International, 2021), allowing drivers to engage in non-driving related tasks (NDRT), such as playing video games and watching films (Sun et al., 2021). However, when an ADS reaches its operational limit, it will prompt human drivers to resume control of the vehicle, known as takeovers.

Scheduled Takeovers and Takeover Requests

McCall’s taxonomy classifies takeovers into scheduled and nonscheduled takeovers, depending on whether the driver is forewarned about the takeover event (McCall et al., 2019). Nonscheduled takeovers occur in unexpected situations, such as road construction and obscured traffic signs (Körber et al., 2018). Conversely, scheduled takeovers are anticipated events, typically happening at predictable instances like freeway exits (Worle et al., 2020).

Scheduled takeovers can be prevalent because they occur in both L3 and L4 autonomous driving. According to SAE International, 2021, at L3, drivers are mandated to serve as a safety backup, whereas, at L4, it becomes optional as L4 ADSs can ensure minimal risk conditions within their defined ODD. However, in both L3 and L4, the ADS will notify drivers to take over when it approaches the exit of its ODD. Furthermore, advancements in information technologies enable ADSs to predict unexpected situations well in advance (Weaver & DeLucia, 2020), potentially reshaping current nonscheduled takeover scenarios into scheduled ones.

In comparison to nonscheduled takeovers, scheduled ones involve longer lead times for takeover requests (TORs). Studies that compared nonscheduled and scheduled takeovers set TOR lead times at 5–6 seconds for the nonscheduled scenarios and extended to 15–50 seconds for the scheduled situations (Pampel et al., 2019; Wu et al., 2019; Yun & Yang, 2020). In addition to the single-stage alerts, researchers have proposed a two-stage approach that incorporates an extended prewarning stage to enhance drivers’ readiness for taking over (Ma et al., 2020; van der Heiden et al., 2017). During the prewarning stage, drivers are provided with information about an upcoming takeover event but are not immediately required to take over.

Holländer and Pfleging (2018) implemented a two-stage alerting system in their scheduled takeover study where drivers received a pre-alert 1 minute before the TOR, and the TOR itself was issued 10 seconds before reaching the exit. Similarly, Hecht et al. (2022) used ambient colors to notify drivers about the impending takeover 150 seconds in advance, subsequently delivering TORs repeatedly at intervals of 28, 14, and 7 seconds before reaching the system’s limit.

NDRT Disengagement

Unlike dual-task situations in traditional manual driving, drivers can immerse themselves in NDRTs during L3 and above automated driving (Ko & Ji, 2018). However, when they need to take over, transitioning from NDRTs to driving represents a typical task-switching paradigm, in which subjects exhibit delayed responses and diminished performance, known as the switching cost (Monsell, 2003). However, with extended preparation time between tasks, the switching cost is notably reduced (Meiran et al., 2000).

Despite the extended preparation time in scheduled takeovers, drivers may not utilize it optimally. Studies indicate that drivers exhibit significantly delayed response in noncritical takeover scenarios, such as freeway exits (Eriksson & Stanton, 2017; Pampel et al., 2019). Wandtner et al. (2018) introduced a prewarning stage preceding the TOR and expected drivers to promptly disengage from NDRTs upon request due to the advanced notice. However, 65% of the participants continued with their NDRTs after receiving the TOR. Additionally, Hecht et al. (2022) detailed the disengagement timings (DTs) observed among participants during scheduled takeovers with two-stage alerts: 26.67% disengaged from NDRTs during the prewarning stage, 70.47% between the TOR and the moment of taking over, and 2.86% after taking over. These studies shed light on potential safety concerns associated with drivers’ disengagement from NDRTs during scheduled takeovers.

Wickens (2002) identified three distinct types of activities that utilize limited processing resources in the human brain: (1) perceptual activities involving the acquisition of specific information from various sensory channels; (2) cognitive activities encompassing higher-level thinking processes like contemplation, reflection, memory retention, and planning; and (3) responding activities involving the conscious selection and execution of responses, including verbal and physical reactions. Building on these three activity types, we propose an operational definition of NDRT disengagement:

NDRT Disengagement Entails the Cessation of Perceptual, Cognitive, and Responding Activities Related to NDRT Within a Defined Timeframe

Previous studies have reported drivers’ disengagement from NDRTs based on the observed eyes or hands diversion (Lin et al., 2019; Mok et al., 2017; Wandtner et al., 2018). However, explicitly ending a task does not necessarily indicate the termination of relevant cognitive activities (Logan, 1985), which has been overlooked in prior studies. Furthermore, NDRT disengagement in scheduled takeovers is not a transient response to TORs but rather a prolonged process starting from the prewarning phase. There is a lack of comprehensive study on NDRT disengagement that considers its associated cognitive activities, examines its impact on takeover performance, and investigates its process in scheduled takeovers.

Early disengagement from NDRTs allows for more preparation time and may enhance takeover performance. A limited number of studies have supported this notion. For instance, Wandtner et al. (2018) reported that participants who postponed NDRT disengagement exhibited compromised lateral control performance compared to their baseline in the without-NDRT condition, while others did not, indicating an interaction between NDRT and DTs. Additionally, although Hecht et al. (2022) did not investigate the influence of DTs on takeover performance, they did consider early NDRT disengagement as a default safer option. Therefore, we propose:

Hypothesis 1 (H1)

Timings of disengagement from NDRTs moderate the impact of NDRTs on takeover performance.

Studies have observed instances where drivers reengage in NDRTs after their first gaze on the road during takeovers (Tan & Zhang, 2022; Xu et al., 2022). Reengagement in an NDRT was seen as an indication of a participant prioritizing the NDRT (Tan & Zhang, 2022), potentially reducing attention allocated to preparing for takeovers (Xu et al., 2022). These studies adopted a single-stage alert, and they predominantly emphasized the negative aspects of drivers’ reengagement in NDRTs. However, in scheduled takeovers with a two-stage alerting system, visual disengagement and reengagement with NDRTs can occur during prewarning and have different implications.

Zeeb et al. (2015) classified drivers into high, medium, and low-risk groups based on their visual behaviors during autonomous driving prior to the TOR. Their research showed that less visual engagement in NDRTs and more focus on the road correlated with a lower risk level and a quicker response to TORs. In line with this notion, we propose that temporary visual disengagement from NDRTs during the prewarning stage indicates reduced attention toward the NDRT and may contribute to earlier eventual disengagement.

Hypothesis 2 (H2)

Drivers who demonstrate more instances of averting their gaze from NDRT during the prewarning stages are expected to achieve an earlier final NDRT disengagement during scheduled takeovers.

The Current Study

The primary objective of this study was to comprehensively examine when drivers disengage from NDRTs during scheduled takeovers with two-stage alerting systems and how DTs affect takeover performance. Furthermore, the study aimed to establish a link between visual disengagement from NDRTs during the prewarning stage and the eventual DTs. These objectives were investigated through driving simulator-based experiments, in which the NDRT disengagement process of participants was observed from visual, physical, and cognitive perspectives.

Methods

Participants

A total of 37 drivers with valid driver’s licenses, comprising 15 females and 22 males, participated in this study. Their ages ranged from 18 to 52 years old (Mean = 30.2, SD = 8.43), and their driving mileage varied from 5000 km to 400,000 km (Mean = 101,500 km, SD = 104,200 km). None of the participants had prior experience with Level 3 or higher automated driving. However, 11 participants had previous exposure to lower-level driver assistance features, such as adaptive cruise control and lane-keeping assistance. As a token of appreciation for their participation, participants who completed the study were rewarded with a gift valued at 100 RMB.

This study received approval from the Ethics Committee at the University of Nottingham Ningbo, China, ensuring adherence to ethical guidelines. Informed consent was obtained from all participants through a signed consent form.

Apparatus

In this empirical study, a driving simulator was employed (Figure 1), utilizing the bodyshell of a Chery QQ car. The original steering wheel, brake, and instrument platform were replaced with a Logitech G29 steering control and pedal set. The G29 steering wheel was customized to a larger size, measuring 14 inches. The driving environment was simulated using CARLA (Dosovitskiy et al., 2017) and was displayed across three screens surrounding the car bodyshell. Additionally, a camera was positioned on the rear-right-hand side of the driver’s seat to monitor driver behavior.

Figure 1.

The driving simulator.

Tobii Pro2, a head-mounted eye-tracking device, was utilized to monitor drivers’ eye movements (Figure 2). This system operated at a detection rate of 50 HZ, with an accuracy of .5 degrees of visual angle.

Figure 2.

The eye-tracking device.

Experimental Design

The experiments employed a within-subject design with two conditions: one involving an NDRT and another as a baseline without an NDRT. Participants completed both trials, and the order was counterbalanced to mitigate the potential learning effect. To further counteract potential biases, the driving directions were reversed between the two test trials, as illustrated in Figure 3. Each trial involved 4 minutes of autonomous driving, followed by approximately 2 minutes of manual driving.

Figure 3.

Driving route (a) First trial; (b) second trial.

The autonomous driving phase unfolded along an eight-lane road, split into four lanes in each direction. To enhance the simulation’s realism, we generated twenty vehicles positioned around the ego vehicle at the starting point. During the takeover period, four vehicles ran in the two lanes to the left of the ego vehicle, but no vehicle appeared in the ego and right lanes.

The scheduled takeover scenario was set as exiting an overpass, following a setup used in prior studies (e.g., Holländer & Pfleging, 2018; Worle et al., 2020). Drivers could take over by depressing the accelerator or brake pedal or turning the steering wheel. After taking over, participants were instructed to transition to the right adjacent lane, and then exit via a circular single-lane ramp (Figure 4).

Figure 4.

Takeover scene.

This study simulated an L4 ADS, obviating the necessity for abrupt takeovers during autonomous driving. According to SAE International, 2021, L4 ADSs can execute operations with minimal risk if drivers cannot take over. However, there has yet to be a consensus regarding the definition of minimal-risk conditions for takeovers at freeway exits. In this study, if drivers failed to take over within the given time, the vehicle would proceed beyond the exit and pull over to the roadside. This approach was inspired by strategies observed in previous studies, such as continuous driving along the freeway (Holländer & Pfleging, 2018) and stopping in the current lane (Worle et al., 2020).

This study employed a two-stage alerting system, comprising a pre-alert sent 1 minute before the TOR, followed by the TOR issued 20 seconds before reaching the exit. The pre-alert provided drivers with advanced information about the upcoming situation without requiring an immediate takeover response. Both audio messages and visual cues were employed in the alerts, presented in Chinese. The audio messages were delivered in a male voice, while the visual cues were displayed over the driving scene. Table 1 shows more details on the translated contents of the alerts, while Figure 5 contains a screenshot depicting the visual cues.

Table 1.

Two-stage Alerting System.

	Pre-alert > (60s) >	TOR > (20s) >	Stop
Audio:	“Reaching the exit of qualified road section for autonomous driving in 1 minute.”	“Please takeover.”	“Vehicle stopped, please drive away soon.”
Visual:	“Reaching the exit of qualified road section after (count down from 60) seconds.”	Hands-on steering wheel image.	Hands-on steering wheel image.
Visual:		“Please takeover in (count down from 20) seconds.”	Hands-on steering wheel image.

Figure 5.

Screenshot of visual cues projected to the scene and the English translation.

The NDRT chosen for this study was a word puzzle game designed to solve six clues using no more than two one-word answers (see Appendix 1). These clues were printed on an A4-sized paper and affixed to a cardboard support. Participants were provided with a pen for note-taking. A holder for the NDRT materials was placed adjacent to the steering wheel, allowing participants to easily store the props when transitioning to a takeover scenario.

The selection of this particular NDRT was motivated by its ability to engage all three types of resources under investigation: visual, cognitive, and physical. Additionally, the word puzzle task offered the added advantage of inducing sustained task-related thoughts (Leroy, 2009). The word puzzle task was designed with an unsolvable difficulty level within the allocated time to ensure that it would not be completed before the takeover. Moreover, participants were informed that upon completing the experiment, they would be required to explain their answers and thought processes. This procedure was designed to enhance participants’ motivation to solve the word puzzle by leveraging their innate sense of fulfillment derived from accomplishing a task (Hennessey et al., 2015) and their yearning for acknowledgment of their competence (Reiss, 2004).

Visual and Physical Disengagement Recording

The word puzzle task required participants to engage in both visual perception and manual operation. Instances where participants averted their eyes and withdrew their hands from NDRTs were labeled as visual and physical disengagement, respectively. To track these movements, eye-tracking glasses and a camera continuously recorded participants’ eye and hand motions.

The eye-tracking and camera videos were synchronized using ErgoLAB 3.0 (see Figure 6), an ergonomics analysis software. Periods of “eye-off-NDRT” and “hands-off-NDRT” were marked when gaze points moved outside the NDRT prop, and when subjects were not holding the props, respectively.

Figure 6.

Synchronization of visual and physical disengagement records.

Cognitive Disengagement Recording

Cognitive disengagement data were collected through a method known as eye-tracking-stimulated recall. This approach employs recorded gaze videos to prompt participants to recall their thoughts during a specific event, providing insights into their cognitive processes (Lyle, 2003; Schindler & Lilienthal, 2020). Eye-tracking stimulated recall is particularly suitable when the task being studied does not involve explicit body movement (Bicer & Bicer, 2022).

Participants were shown the eye-tracking videos immediately after the with-NDRT trial. These videos included a segment from 10 seconds before the pre-alert until the conclusion of the experiment. The videos were paused at three specific instances: (1) when the ADS was deactivated, (2) when the ego vehicle entered the exit, and (3) 10 seconds after the previous pause. At these pauses, participants were prompted to indicate whether they had thoughts related to the NDRT in the last 10 seconds shown in the video, responding either “yes” or “no.”

To aid participants in identifying NDRT-related thoughts, we provided them with examples based on a questionnaire developed by Newton et al. (2020). These examples included tasks such as seeking answers for the word puzzle task, planning for further actions related to the word puzzle task, and reflecting on past actions. Participants’ responses and the moments of taking over and entering the exit, were documented on a record form (see Figure 7).

Figure 7.

Example of cognitive disengagement records.

Pilot Test for Validating the Method for Recording Cognitive Disengagement

While unconventional for labeling quantitative data, the stimulated recall method was the most viable approach given the study’s unique requirements. We aimed to dynamically evaluate participants’ cognitive activities related to the NDRT without interrupting the takeover process. Although automatic classifiers have been developed to detect drivers’ mind-wandering using machine learning methods, achieving optimal accuracy remains a challenge, with reported accuracies ranging from around 60% (Jin et al., 2019) to 67% (Chen et al., 2022). Furthermore, thoughts unrelated to driving may not necessarily be linked to NDRTs.

Thought probing, a well-established method for ascertaining subjects’ thoughts during a specified period (Weinstein, 2018), has been frequently employed in studying drivers’ mind wandering (e.g., Zhang & Kumada, 2018). However, it necessitates participants to report their attention allocation when prompted, interrupting the primary task. In our study, thought probing was utilized to validate the reliability of the results obtained through eye-tracking stimulated recall.

To compare the results obtained from thought probing and eye-tracking stimulated recall, we conducted a pilot test involving twenty participants. This test replicated the training and trial phases detailed in the “Procedures” section but included additional thought-probing operations. The auditory probes read “Any word puzzle thoughts?” for a duration of .8 seconds. Participants were instructed to respond to thought probes by verbally indicating “yes” or “no” based on whether they had thoughts related to the word puzzle task when receiving a probe.

Six probes were automatically delivered at specific times: (1) 20 seconds after the pre-alert, (2) 40 seconds after the pre-alert, (3) 10 seconds after participants took over, (4) 10 seconds after participants entered the ramp, (5) 30 seconds after participants entered the ramp, and (6) 60 seconds after participants entered the ramp. Participants’ responses to the probes were documented on a record form (see Figure 8). Following the tests, eye-tracking stimulated recalls were conducted as previously described. The audio on the eye-tracking video was intentionally muted during playback to ensure participants’ responses were not influenced by any auditory cues or associated memories.

Figure 8.

Example of the thought probing records (“p#” indicates the probe number).

The thought probing technique was employed to label participants’ cognitive status during the 10 seconds prior to the probes, following a well-established approach used in previous studies (He et al., 2011; Zhang & Kumada, 2018). This approach allowed for seamless pairing with the data obtained from eye-tracking stimulated recall, resulting in 113 data pairs.

To validate the congruence between eye-tracking stimulated recall and thought probing, we analyzed the paired data using the Kappa test performed with SPSS version 28.0. The analysis revealed excellent agreement between eye-tracking stimulated recall and thought probing, indicated by a high Kappa value of .928 (p < .001) (see Table 2). This result strongly suggests that eye-tracking stimulated recall is a reliable method for determining whether participants have thoughts related to the NDRT during a specific period.

Table 2.

Results From Thought Probing and Eye-tracking Stimulated Recall.

		Participants’ Disengagement Based on Eye-tracking Stimulated Recall		Total
		No	Yes	Total
Participants’ disengagement based on thought probing	No	47	3	50
Participants’ disengagement based on thought probing	Yes	1	62	63
Total		48	65	113

Posttest Interview

Upon completing both tests, participants were provided with the definitions of visual, cognitive, and physical disengagement. Subsequently, they were instructed to indicate on a timeline, as illustrated in Figure 9, the point at which they deemed it necessary to disengage from NDRTs to ensure a safe takeover. Afterward, a researcher showed them their disengagement records. If the disengagement time exceeded the participants’ voting, they were required to explain the discrepancies.

Figure 9.

Vote Collection form.

Procedures

Upon arrival at the laboratory, participants were provided with information about the features of the simulated ADS, the rules of the word puzzle task, and the operation of the driving simulator. Since stopping on highways is unsafe, clear instructions were given that they were expected to take over, despite the system being programmed to pull the vehicle over if drivers took no action. Participants were reminded to adhere to real-life driving regulations and exercise caution, as their performance would be evaluated. However, they had the liberty to decide whether and when to disengage from the NDRT, given their potential ability to drive and conduct NDRTs concurrently.

Participants received takeover training on a route identical to the one depicted in Figure 3(a), with the only difference being that the commencement of autonomous driving was set to 5 seconds before the pre-alert. After the takeover training, participants had a three-minute session for free driving to acquaint themselves with the simulator. The trials began once participants confirmed their ability to maintain stable vehicle control. Upon completing the with-NDRT trials, participants’ cognitive disengagement was assessed using the eye-tracking-stimulated recall method detailed in the “Cognitive Disengagement Recording” section.

Dependent Variables

Table 3 presents the metrics employed in this study to assess participants’ takeover performance and visual disengagement during the prewarning stage.

Table 3.

Metrics Used in This Study.

	Metrics	Description
Takeover performance	Takeover time (TOT)	The interval between the pre-alert and the moment of deactivating the autopilot.
	Maximum lateral acceleration (MAlat)	Maximum lateral acceleration (MAlat) and maximum longitudinal acceleration (MAlon) during the takeover period.
	Maximum longitudinal acceleration (MAlon)
Visual disengagement during the prewarning stage	Eye-off-NDRT duration (EoD)	The total duration the drivers had their eyes off the NDRT during the prewarning stage.
Visual disengagement during the prewarning stage	Eye-off-NDRT count (EoC)	The number of times drivers switch their eyes between the NDRT props and other areas during the prewarning stage.

(Cao et al., 2021) categorized takeover performance metrics into two main aspects: time and quality. In this study, we adopted TOT to quantify the time aspect, while MAlat and MAlon were utilized to assess the quality aspect. Generally, shorter TOT is preferable, affording drivers more time to execute the takeover (Dogan et al., 2019; Ma et al., 2020). Conversely, smaller MAlat and MAlon are sought after as they indicate a more controlled and stable takeover maneuver (Lee et al., 2020; Ma et al., 2020).

The assessment of visual disengagement during the prewarning stage was derived from the disengagement recordings outlined in previous sections. Notably, these measurements exclusively pertain to the with-NDRT trials, as NDRT disengagement was not a factor in the without-NDRT trials.

Data Analysis

One subject’s data was excluded from the analysis as the participant failed to take over in time during the with-NDRT trial. Despite the option for drivers to decline taking over and maintain a minimal-risk condition with the L4 ADS, this specific participant was omitted due to the study’s primary focus being on investigating takeover performance and NDRT disengagement.

The mixed analysis of variance (ANOVA) was employed to test H1, using NDRT (with and without NDRT) as the within-subject factor and DTs (DT1, DT2, and DT3) as the between-subject factor. The assumption of sphericity was automatically met because the within-subject factor had only two levels. To ensure the absence of salient outliers in any subgroup involved in the comparisons, data from seven participants were excluded. An outlier was defined as a data point whose deviation from the median exceeded three times the interquartile range.

Subsequently, the Shapiro–Wilk and Levene tests were conducted to assess the normality and variance homogeneity of the remaining data in each subgroup (Table 4). Most results met the criteria, with “p” values exceeding .05. There were two exceptions with “p” values of .032 and .021, slightly below the significance level. Given the robustness of the Mixed ANOVA method, we decided to proceed with the analysis without further data transformation.

Table 4.

Test for Normality and Homogeneity of Variance.

			TOT		MALat		MALong
			Statistic	p	Statistic	p	Statistic	p
Normality	With-NDRT	DT1	.871	.153	.857	.112	.947	.684
		DT2	.926	.409	.923	.379	.952	.696
		DT3	.954	.691	.895	.158	.932	.431
	Without-NDRT	DT1	.925	.472	.959	.798	.834	.066
		DT2	.898	.209	.846	.052	.939	.545
		DT3	.881	.106	.840	.032	.940	.521
Homogeneity of variance	With-NDRT		3.220	.056	4.510	.021	1.124	.340
Homogeneity of variance	Without-NDRT		1.271	.297	2.390	.111	1.539	.234

Recognizing the limitation resulting from omitting outliers, we retained them in the subsequent analyses. A comparison of takeover performance in the with and without NDRT conditions was conducted to investigate the impact of NDRT engagement on scheduled takeovers. Subsequently, participants were categorized into three groups based on their NDRT disengagements timings in the with-NDRT trials. The comparison of takeover performance was performed within each group separately to study the effect of NDRT under varying DTs. Due to the nonnormal distribution of the data, the Wilcoxon Signed-rank Test was employed. Furthermore, comparisons between each pair of the three groups were conducted using the Kolmogorov–Smirnov test.

The Kruskal–Wallis Test was employed to compare visual disengagement during the prewarning stage across the three groups, while pairwise comparisons were carried out using the Kolmogorov–Smirnov test. All analyses were performed using SPSS 28.0, with a significance level set at .05.

The audio recordings of follow-up interviews were transcribed verbatim and analyzed using thematic analysis (Braun & Clarke, 2006). Researchers thoroughly reviewed the texts, identifying meaningful text segments explaining participants’ disengagement behaviors. Initial themes, representing key concepts, were derived and used to code the transcribed data in NVivo 12, with iterative updates until team consensus was reached. The final version of themes and coding rules is detailed in Table 5’s codebook. An external researcher with expertise in human–computer interaction utilized this codebook to code the raw data. The intercoder reliability (ICR) test yielded a Cohen’s kappa coefficient of .82, indicating a high level of reliability (Burla et al., 2008).

Table 5.

Codebook for Thematic Analysis of Interview Results.

Themes	Coding Criteria
Traffic environment	Participants mentioned attributes of the traffic environment, such as density, weather, and road curvature, or evaluated its demand level.
Motivation towards continuing the NDRT	Participants expressed their motivation towards the NDRT or commented on NDRT features that induced their motivation.
Attentional control	Participants expressed incapability to control cognitive activities.
Tacking the time pass	Participants expressed their lack of awareness of the remaining time before takeovers.
Distracted driving habits	Participants referenced their previous distracted driving experiences.
Experiment nature	Participants commented on characteristics unique to the experimental setup that do not typically occur in daily life scenarios.

Results

Identification of NDRT Disengagement Timings

Participants were divided into three groups based on their DTs during the with-NDRT condition. The operational definition proposed in this study delineated DTs according to when participants achieved the ultimate stages of visual, cognitive, and physical disengagement throughout the takeover phase. The identified DT groups are as follows (“N” represents the number of participants in each group):

DT 1. Disengaged from the NDRT prior to the TOR (N = 12).

DT 2. Disengaged from the NDRT before taking over but after TOR (N = 11).

DT 3. Continued NDRT (Not disengaged) after taking over (N = 13).

For each group, NDRT disengagement recordings were graphically plotted on a timeline, as illustrated in Figure 10(a)–(c). These figures represent the percentage of participants who maintained engagement with the NDRT from the three perspectives throughout the takeover period, from the issuance of the pre-alert to the moment of entering the ramp. The timelines within the figure depict the countdown of the pre-alert and the TOR.

Figure 10.

(a) NDRT disengagement recordings of the DT1 group. (b) NDRT disengagement recordings of the DT2 group. (c) NDRT disengagement recordings of the DT3 group.

Takeover Performance

Mixed ANOVA analysis showed a significant interaction between NDRT involvement and DTs in terms of their effect on takeover time (F (2, 30.9) = 4.495, p = .021, η² = .257), but not on the takeover quality metrics (i.e., MALat and MALong). These results partially confirmed our hypothesis that timings of disengagement from NDRTs moderate the impact of NDRTs on takeover performance (H1).

Table 6 presents the within-subject comparisons using the dataset retaining the outliers. Initially, the comparison before grouping exposed significant adverse effects of the NDRT on TOT, while no notable difference was observed in takeover quality. Subsequent separate comparisons conducted within the three DT groups yielded diverse results, which contribute additional evidence in support of H1.

Table 6.

Comparison Between With-NDRT and Without-NDRT Conditions.

		With-NDRT		Without-NDRT		p	z
		Mean	SD	Mean	SD	p	z
Overall	TOT (s)	65.365	7.789	62.240	8.442	<.001	−3.323
	MALat (m/s²)	1.422	1.744	.854	.457	.076	−1.775
	MALong (m/s²)	4.599	3.879	5.821	4.042	.062	−1.870
DT1	TOT (s)	58.685	8.963	57.356	12.792	.666	−.432
	MALat (m/s²)	1.264	2.236	.807	.379	.480	−.706
	MALong (m/s²)	4.238	3.652	6.504	3.602	.050	−1.961
DT2	TOT (s)	69.206	3.796	65.185	2.997	.013	−2.490
	MALat (m/s²)	1.418	1.161	.994	.599	.213	−1.245
	MALong (m/s²)	4.197	3.837	5.089	3.502	.594	−.533
DT3	TOT (s)	68.281	4.895	64.256	3.763	.009	−2.621
	MALat (m/s²)	1.573	1.764	.779	.391	.023	−2.271
	MALong (m/s²)	5.271	4.316	5.810	4.958	.807	−.245

In the DT1 group, NDRT involvement did not significantly hinder participants’ takeover performance. On the contrary, participants demonstrated notably smaller MALong values in the with-NDRT condition, signifying improved longitudinal control performance. Within the DT2 group, the results revealed a significant difference between the with-NDRT and without-NDRT conditions only in terms of TOT. Regarding participants displaying DT3, their takeover performance in the with-NDRT condition was significantly worse, except for the MALong.

Table 7 shows the comparison of participants’ takeover performance across the three groups. The comparison between DT1 and DT2 unveiled a significant disparity in takeover time for both with and without NDRT conditions. Further comparisons between DT1 and DT3 groups showed noteworthy differences in TOT and MALat in the presence of NDRT. Lastly, no significant difference was found between the DT2 and DT3 groups.

Table 7.

Comparison Between DT Groups on Takeover Performance Metrics.

		With-NDRT		Without-NDRT
		p	z	p	z
DT1-DT2	TOT (s)	<.001	2.396	.049	1.361
	MALat (m/s²)	.045	1.379	.928	.544
	MALong (m/s²)	.945	.526	.607	.762
DT1-DT3	TOT (s)	.001	1.906	.334	.945
	MALat (m/s²)	.026	1.473	.650	.737
	MALong (m/s²)	.966	.496	.677	.721
DT2-DT3	TOT (s)	.625	.751	.767	.666
	MALat (m/s²)	.956	.512	.486	.836
	MALong (m/s²)	.909	.563	.845	.615

Visual Disengagement During the Prewarning Stage

Figure 11 compares average eye-off-NDRT duration (EoD) and count (EoC) across the three groups within six time windows during the prewarning stage. In general, the DT1 group demonstrated the longest EoD and highest EoC among the three groups. In contrast, the DT2 group had longer EoDs than the DT3 group but lower EoCs. The Kruskal–Wallis Test revealed significant differences among the three groups in EoD during the 50s and 60s windows and EoC during the 30s, 40s, and 50s windows. Further pairwise comparisons results indicated significant differences in EoD and EoC for the 60s window between DT1 and the other two groups, but no notable difference was found between DT2 and DT3 (Table 8).

Figure 11.

Comparison of EoD and EoC across the three groups in different windows (Error bars: 95% confidence interval. “H”: Kruskal–Wallis H value).

Table 8.

Pairwise Comparisons on EoD and EoC Through the Kolmogorov–Smirnov Test.

		EoD		EoC
		50s Window	60s Window	30s Window	40s Window	50s Window
DT1-DT2	p	.091	.018	.284	.069	.025
DT1-DT2	z	1.244	1.536	.988	1.298	1.481
DT1-DT3	p	.074	.037	.846	.242	.456
DT1-DT3	z	1.453	1.513	.613	1.027	.856
DT2-DT3	p	.995	.921	.397	.783	.269
DT2-DT3	z	.414	.552	.897	.656	1.001

Posttest Interview

Votes on the perceived safe timing of NDRT disengagement were classified into three groups, aligning with the three DTs elucidated earlier. As outlined in Table 9, the majority of votes (23 out of 36) concurred with DT1. In contrast, two participants expressed the view that sustained cognitive engagement in NDRTs (DT3) might not necessarily affect their driving. Shaded cells highlight instances where participants exhibited delayed NDRT disengagement compared to their votes. Table 10 presents the six themes that surfaced from the thematic analysis of the interview data, each supported by an illustrative quote.

Table 9.

Comparison Between Voting Results and the Actual DT in Experiments.

Table 10.

Participants’ Explanations for Disengaging Later Than Their Voted Safe Disengage Time.

Themes. (Number of Results)	Quotes
Traffic environment. (6 from DT 2; 3 from DT3)	“There was no car on the road at the time, so it did not require much attention.”
Motivation towards the NDRT. (3 from DT 2; 2 from DT3)	“I was afraid to be the only one who failed to figure out the word puzzles.”
Attention control. (5 from DT 3)	“It was hard to switch my attention away from the NDRT because it was not finished.”
Tacking the time pass. (3 from DT2; 1 from DT 3)	“I did not realize the time passage until receiving the takeover request.”
Distracted driving habits. (3 from DT 3)	“I usually think about something while driving.”
Experiment nature. (4 from DT2, 1 from DT3)	“I dared to do that because there is no threat to my safety in this experiment.”

Discussion

Identification of NDRT Disengagement Timings

In line with previous studies that utilized a two-stage alerting system for scheduled takeovers, this study confirms the presence of varying disengagement timings (DTs) among drivers (Hecht et al., 2022). The distribution of DTs was observed to be proportionally spread among DT 1, DT 2, and DT 3 at 33.3%, 30.6%, and 36.1%, respectively. This finding underscores the need to examine the effects of DTs and design interventions to regulate them accordingly. While DT3 exhibited the highest proportion in this study, it accounted for only 2.86% in a prior study (Hecht et al., 2022), and no such case was reported by Wandtner et al. (2018). This discrepancy is understandable, considering previous studies did not delve into the cognitive facet of NDRT disengagement.

In this study, participants who exhibited DT3 disengaged from the NDRT visually and physically before taking over, but not cognitively. However, failures in visual and physical disengagement are also plausible, as evidenced by instances of drivers engaging in distraction activities such as texting while driving (Harrison, 2011; Liang et al., 2015). According to the multiple resource model, the difficulty of simultaneously performing two tasks depends on the level of task demand and the overlap in resources they consume (Wickens, 2002). Pure cognitive activities overlap less with driving compared to visual and motoric activities, potentially explaining why cognitive disengagement failure appeared to be more likely in this study.

As depicted in Figure 9, all three groups exhibited delayed cognitive disengagement compared to visual and physical disengagement. Previous research considered drivers as disengaged from NDRTs once their eyes and hands are no longer engaged with the task (e.g., Lin et al., 2019; Mok et al., 2017). However, our study unveils that even when drivers have averted their eyes and hands from NDRTs, they might remain cognitively engaged. In the cases of DT1 and DT2, delayed cognitive disengagement could impede drivers’ readiness for takeovers due to the occupation of cognitive resources.

The delayed cognitive disengagement in cases of DT3 resembles a state of cognitive distraction for drivers. Although pure cognitive distraction has little impact on lane-keeping performance (Cao & Liu, 2013), it is associated with longer response times to sudden events (Yanko & Spalek, 2014). Given that drivers’ situational awareness is often compromised during takeovers compared to traditional manual driving, it becomes crucial to ensure that drivers disengage cognitively from NDRTs after resuming vehicle control.

Analysis of Takeover Performance

This study compared participants’ takeover performance under conditions with and without NDRT. The results revealed significant delays in participants’ TOT in the presence of NDRT, consistent with previous studies on noncritical takeovers at freeway exits (Eriksson & Stanton, 2017; Naujoks et al., 2019). However, the influence of the NDRT on takeover quality did not reach a significant level.

The impact of NDRTs on takeover performance varied based on the disengagement timings, aligning with Wandtner et al. (2018), who reported negative effects associated with delayed NDRT disengagement. In this study, participants demonstrating DT3 were significantly affected by the NDRT in both takeover time and quality, whereas no significant adverse impact from the NDRT was observed for participants displaying DT1. Moreover, the between-subject comparison revealed significantly better takeover performance in the DT1 group than in the other two. This result may be attributed to participants in the DT1 group having more time to resume their situation awareness after disengaging from the NDRT. NDRT engagement has been reported to impair drivers’ situation awareness during takeovers (Marti et al., 2022), whereas rebuilding situation awareness can be a time-consuming process (Lu et al., 2017). Therefore, this study advocates for early disengagement from NDRTs during the prewarning stage in scheduled takeovers.

The DT3 group showed significantly diminished lateral control quality (MAlat) compared to their baseline, while the longitudinal control (MAlon) remained relatively unaffected. This finding aligns with the research conducted by Engstrom et al. (2017), who demonstrated that cognitive distractions have a selective impact on driving performance. Specifically, they found that automatic driving actions are unaffected by cognitive distractions, whereas actions that require cognitive control are impaired. Compared to Malon, MAlat is a more relevant performance measure for lane changing, which is nonautomatic and can be affected by cognitive load (Engstrom et al., 2017; Reimer et al., 2013).

Visual Disengagement During the Prewarning Stage

Participants frequently displayed temporary visual disengagement from the NDRT during the prewarning stage. Notably, the DT1 group exhibited prolonged EoD and a higher EoC compared to the other two groups. These differences became significant after 30 and 40 seconds, respectively, following the pre-alert, confirming H2. Behavioral and physiological data from the early stages have been employed in predicting drivers’ future takeover behaviors (Tan & Zhang, 2022; Wu et al., 2021), while our findings suggest the potential utility of EoD and EoC as predictors for the eventual DTs. However, no significant difference was observed between the DT2 and DT3 groups concerning EoD or EoC, underscoring the challenge of distinguishing these two DTs. Therefore, further research is needed to explore distinguishable factors of DT2 and DT3 in scheduled takeovers.

Tan and Zhang (2022) argued that when drivers look back at NDRTs after initially glancing at the road, they prioritize the NDRT over driving at that moment. However, we propose that frequent visual transitions between the NDRT and the road reflect drivers’ concerns for both driving and the NDRT. Given that the DT1 group showed the highest EoC, a high EoC can indicate a greater likelihood of early NDRT disengagement.

Posttest Interview

Interviews revealed that participants delayed NDRT disengagement primarily due to their perception of low driving demand. While drivers can regulate their NDRT interaction based on task demands (Liang et al., 2015; Tivesten & Dozza, 2015), this self-regulation may not always be effective, as exemplified by accidents on roads familiar to drivers (Intini et al., 2018). Another frequently cited factor, motivation towards the NDRT, can vary depending on the task and situation (De Brabander & Martens, 2014; Malmberg et al., 2015). In line with Wandtner et al. (2018), this study underscores the importance of examining the motivational aspects of NDRTs in scheduled takeovers.

Five participants who demonstrated DT3 reported challenges in attention control, which refers to the ability to regulate information processing based on the current primary goal (Burgoyne & Engle, 2020). Their delayed cognitive disengagement was not intentional but rather a result of these challenges. Prior studies on the task-switching paradigm have suggested that previous tasks can linger in working memory, hindering the swift transition of attention to a new task (Shipstead et al., 2016). In addition, four participants who initially voted for DT1 but later displayed DT2 and DT3 admitted they failed to notice the passage of time until the TOR signal. The NDRT occupying the focal vision necessitated a temporary disengagement to monitor the countdown projected in the scene, which some participants struggled to achieve.

Three participants exhibiting DT3 cited their distracted driving habits, which reflect a person’s typical behavior patterns and strongly indicate future actions (Ouellette & Wood, 1998). Hansma et al. (2020) established a connection between drivers’ habitual cell phone usage in nondriving contexts and their engagement in cell phone use while driving. Similarly, habits formed during traditional manual driving may carry over into takeover situations.

Interviews disclosed that some participants delayed NDRT disengagement due to the simulator-based nature of the experiment, which inherently lacks safety threats. Despite this limitation, driving simulators have proven to be valuable in studying drivers’ takeover behaviors, as evidenced by previous research (e.g., Eriksson & Stanton, 2017; Roche et al., 2019). Furthermore, the simulator used in this study, featuring a car body and a projected immersive driving scene, enhanced realism (Helman & Reed, 2015), and it has been successfully applied in a previous takeover study (Sun et al., 2021).

Implications

First, our results highlight that takeover performance remained relatively unaffected by the NDRT when participants employed DT1, emphasizing the importance of promoting NDRT disengagement during the prewarning stage in scheduled takeovers. Second, it is important to acknowledge the potential for delayed or incomplete cognitive disengagement. Therefore, ADSs should consider this when assessing drivers’ readiness for takeover and implementing strategies to facilitate cognitive disengagement. Additionally, variations in visual disengagement during the prewarning stage among the three DT groups suggest the potential use of visual disengagement as a predictor for eventual DTs.

Limitations and Future Study

One important consideration is that participants faced no safety risks while operating the driving simulator. Consequently, they may have exhibited more audacious behavior than they would have in real-world driving scenarios. On the other hand, despite receiving training before the tests, their familiarity with the simulator differed from their personal vehicles, which could have influenced disengagement behaviors and vehicle control performance. In order to address this concern, future research could be conducted using actual vehicles in controlled testing environments, where safety can be ensured while still capturing realistic driving behaviors.

As detailed in the “method” section, we utilized the Eye-tracking-stimulated recall method, which was the most practical choice available. However, this method has inherent limitations, including being time-consuming and lacking the ability to assess the intensity or intentionality of cognitive activities. In future studies, researchers could explore and develop more advanced methods to detect drivers’ cognitive disengagement dynamically and accurately.

Although results may vary under different conditions, we focused on a single traffic and NDRT condition to emphasize the impact of disengagement timings. Future research should broaden the scope of investigation to better understand the relationship between traffic conditions, NDRT attributes, and disengagement timings.

The participants involved in this study were relatively young, potentially limiting the direct applicability to older age groups. Additionally, the sample size was relatively small, especially when divided into three groups. Future researchers should consider increasing the sample size and including a more diverse range of participants to validate and apply our research.

Conclusion

This study investigated the timing of drivers’ disengagement from NDRTs in scheduled takeovers, delineating three distinct disengagement timings: disengaged before the TOR (DT1), disengaged before taking over (DT2), and not disengaged before taking over (DT3). The impact of NDRTs on takeover performance varied among participants, with the DT1 group demonstrating resilience with no adverse effect, the DT2 group experiencing notable impairment on takeover time, and the DT3 group being adversely affected on both takeover time and quality.

Cognitive disengagement appeared to be likely to be delayed, causing incomplete disengagement from NDRTs after takeovers. However, it can be subject to NDRT attributes and takeover conditions, which is worth further investigation. Furthermore, the DT1 group exhibited longer EoD and higher EoC during the prewarning stage compared to the other two groups. However, no significant difference was observed in EoD and EoC between the DT2 and DT3 groups, underlining the challenge of distinguishing these disengagement timings based on the examined visual cues.

Promoting early disengagement and cognitive disengagement from NDRTs emerges as a promising strategy to enhance readiness in scheduled takeovers. Future research and technology advancements should leverage these insights to develop robust driver assistance systems, ultimately contributing to safer automated driving experiences.

Key Points

In scheduled takeovers with a two-stage alerting system, drivers may disengage from NDRTs before the TOR, after the TOR, or not disengage completely.

Participants who disengaged from the NDRT during the prewarning stage demonstrated resilience to the NDRT’s adverse effect on takeover performance.

Facilitating cognitive disengagement from NDRTs is essential given its tendency to be delayed.

Eye-off-NDRT duration and Eye-off-NDRT count during the prewarning stage can be early signs of eventual NDRT disengagement timings.

Footnotes

Acknowledgments

This work is supported by 2025 Key Technological Innovation Program of Ningbo City under Grant No. 2022Z080.

ORCID iDs

Jiming Bai

Xu Sun

Shi Cao

Qingfeng Wang

Jiang Wu

Clues in the Word Puzzle Game

It is a unit.

It is a type of plant.

It has multiple pronunciation.

It is a type of building material.

It is a type of beverage.

It has multiple colors.

Author Biographies

Jiming Bai is a PhD candidate at the Department of Mechanical, Materials, and Manufacturing Engineering, University of Nottingham, Ningbo, China. He received an MSc degree in integrated product design from Delft University of Technology in 2019.

Xu Sun is a professor at the Department of Mechanical, Materials, and Manufacturing Engineering, University of Nottingham, Ningbo, China. She received a PhD degree in human computer interaction from University of Loughborough in 2010.

Shi Cao is an associate professor with the Department of Systems Design Engineering, University of Waterloo. He received a PhD degree in industrial and operations engineering from University of Michigan, Ann Arbor in 2013.

Qingfeng Wang is an associate professor in Economics at the Nottingham University Business School, Ningbo, China. He received a PhD degree in mathematics from University of Loughborough in 2012.

Jiang Wu is an assistant professor in Product Design and Manufacturing, University of Nottingham, Ningbo, China. He received a PhD degree in industrial design from Eindhoven University of Technology in 2020.

References

Bicer

(2022). Understanding young students’ mathematical creative thinking processes through eye-tracking-stimulated recall interview. Mathematics Education Research Journal, 35(2), 361–399. https://doi.org/10.1007/s13394-022-00429-7

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Burgoyne

A. P.

Engle

R. W.

(2020). Attention control: A cornerstone of higher-order cognition. Current Directions in Psychological Science, 29(6), 624–630. https://doi.org/10.1177/0963721420969371

Burla

Knierim

Barth

Liewald

Duetz

Abel

(2008). From text to codings: Intercoder reliability assessment in qualitative content analysis. Nursing Research, 57(2), 113–117. https://doi.org/10.1097/01.NNR.0000313482.33917.7d

Cao

Liu

(2013). Concurrent processing of vehicle lane keeping and speech comprehension tasks. Accident Analysis & Prevention, 59, 46–54. https://doi.org/10.1016/j.aap.2013.04.038

Cao

Zhou

Pulver

E. M.

Molnar

L. J.

Robert

L. P.

Tilbury

D. M.

Yang

X. J.

(2021). Towards standardized metrics for measuring takeover performance in conditionally automated driving: A systematic review. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 65(1), 1065. https://doi.org/10.1177/1071181321651213

Chen

Y. T.

Lee

H. H.

Shih

C. Y.

Chen

Z. L.

Beh

W. K.

Yeh

S. L.

A. Y.

(2022). An effective entropy-assisted mind-wandering detection system using EEG signals of MM-SART database. IEEE Journal of Biomedical and Health Informatics, 26(8), 3649–3660. https://doi.org/10.1109/JBHI.2022.3187346

De Brabander

C. J.

Martens

R. L.

(2014). Towards a unified theory of task-specific motivation. Educational Research Review, 11, 27–44. https://doi.org/10.1016/j.edurev.2013.11.001

Dogan

Honnêt

Masfrand

Guillaume

(2019). Effects of non-driving-related tasks on takeover performance in different takeover situations in conditionally automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 62, 494–504. https://doi.org/10.1016/j.trf.2019.02.010.

10.

Dosovitskiy

Ros

Codevilla

Lopez

Koltun

(2017). Carla: An open urban driving simulator. Proceedings of the 1st Annual Conference on Robot Learning. PMLR. https://proceedings.mlr.press/v78/dosovitskiy17a.html

11.

Engstrom

Markkula

Victor

Merat

(2017). Effects of cognitive load on driving performance: The cognitive control hypothesis. Human Factors, 59(5), 734–764. https://doi.org/10.1177/0018720817690639

12.

Eriksson

Stanton

N. A.

(2017). Takeover time in highly automated vehicles: Noncritical transitions to and from manual control. Human Factors, 59(4), 689–705. https://doi.org/10.1177/0018720816685832

13.

Hansma

B. J.

Marulanda

Chen

H.-Y. W.

Donmez

(2020). Role of habits in cell phone-related driver distractions. Transportation Research Record: Journal of the Transportation Research Board, 2674(12), 254–262. https://doi.org/10.1177/0361198120953157

14.

Harrison

M. A.

(2011). College students’ prevalence and perceptions of text messaging while driving. Accident Analysis & Prevention, 43(4), 1516–1520. https://doi.org/10.1016/j.aap.2011.03.003

15.

Becic

Lee

Y. C.

McCarley

J. S.

(2011). Mind wandering behind the wheel: Performance and oculomotor correlates. Human Factors, 53(1), 13–21. https://doi.org/10.1177/0018720810391530

16.

Hecht

Weng

Kick

L. F.

Bengler

(2022). How users of automated vehicles benefit from predictive ambient light displays. Applied Ergonomics, 103, 103762. https://doi.org/10.1016/j.apergo.2022.103762.

17.

Helman

Reed

(2015). Validation of the driver behaviour questionnaire using behavioural data from an instrumented vehicle and high-fidelity driving simulator. Accident Analysis & Prevention, 75, 245–251. https://doi.org/10.1016/j.aap.2014.12.008.

18.

Hennessey

Moran

Altringer

Amabile

T. M.

(2015). Extrinsic and intrinsic motivation (pp. 1–4). Wiley Encyclopedia of Management. https://doi.org/10.1002/9781118785317.weom110098

19.

Holländer

Pfleging

(2018). Preparing drivers for planned control transitions in automated cars. Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia. Association for Computing Machinery. https://doi.org/10.1145/3282894.3282928

20.

Intini

Berloco

Colonna

Ranieri

Ryeng

(2018). Exploring the relationships between drivers’ familiarity and two-lane rural road accidents. A multi-level study. Accident Analysis & Prevention, 111, 280–296. https://doi.org/10.1016/j.aap.2017.11.013.

21.

Jin

C. Y.

Borst

J. P.

van Vugt

M. K.

(2019). Predicting task-general mind-wandering with EEG. Cognitive, Affective, & Behavioral Neuroscience, 19(4), 1059–1073. https://doi.org/10.3758/s13415-019-00707-1

22.

S. M.

Y. G.

(2018). How we can measure the non-driving-task engagement in automated driving: Comparing flow experience and workload. Applied Ergonomics, 67, 237–245. https://doi.org/10.1016/j.apergo.2017.10.009.

23.

Körber

Prasch

Bengler

(2018). Why do I have to drive now? Post hoc explanations of takeover requests. Human Factors, 60(3), 305–323. https://doi.org/10.1177/0018720817747730

24.

Lee

S. C.

Yoon

S. H.

Y. G.

(2020). Effects of non-driving-related task attributes on takeover quality in automated vehicles. International Journal of Human-Computer Interaction, 37(3), 211–219. https://doi.org/10.1080/10447318.2020.1815361

25.

Leroy

Sophie

(2009). Why is it so hard to do my work? The challenge of attention residue when switching between work tasks. Organizational Behavior and Human Decision Processes, 109(2), 168–181. https://doi.org/10.1016/j.obhdp.2009.04.002

26.

Liang

Horrey

W. J.

Hoffman

J. D.

(2015). Reading text while driving: Understanding drivers' strategic and tactical adaptation to distraction. Human Factors, 57(2), 347–359. https://doi.org/10.1177/0018720814542974

27.

Lin

Liu

Zhang

T. R.

Zhang

(2019). Exploring the self-regulation of secondary task engagement in the context of partially automated driving: A pilot study. Transportation Research Part F: Traffic Psychology and Behaviour, 64, 147–160. https://doi.org/10.1016/j.trf.2019.05.005.

28.

Logan

G. D.

(1985). Executive control of thought and action. Acta Psychologica, 60(2–3), 193–210. https://doi.org/10.1016/0001-6918(85)90055-1

29.

Coster

de Winter

(2017). How much time do drivers need to obtain situation awareness? A laboratory-based study of automated driving. Applied Ergonomics, 60, 293–304. https://doi.org/10.1016/j.apergo.2016.12.003.

30.

Lyle

(2003). Stimulated recall: A report on its use in naturalistic research. British Educational Research Journal, 29(6), 861–878. https://doi.org/10.1080/0141192032000137349

31.

Zhang

Yang

Kang

Chai

Shi

Zeng

(2020). Take over gradually in conditional automated driving: The effect of two-stage warning systems on situation awareness, driving stress, takeover performance, and acceptance. International Journal of Human-Computer Interaction, 37(4), 352–362. https://doi.org/10.1080/10447318.2020.1860514

32.

Malmberg

L.-E.

Pakarinen

Vasalampi

Nurmi

J.-E.

(2015). Students’ school performance, task-focus, and situation-specific motivation. Learning and Instruction, 39, 158–167. https://doi.org/10.1016/j.learninstruc.2015.05.005.

33.

Marti

Jallais

Koustanaï

Guillaume

Mars

(2022). Impact of the driver’s visual engagement on situation awareness and takeover quality. Transportation Research Part F: Traffic Psychology and Behaviour, 87, 391–402. https://doi.org/10.1016/j.trf.2022.04.018.

34.

McCall

McGee

Mirnig

Meschtscherjakov

Louveton

Engel

Tscheligi

(2019). A taxonomy of autonomous vehicle handover situations. Transportation Research Part A: Policy and Practice, 124, 507–522. https://doi.org/10.1016/j.tra.2018.05.005.

35.

Meiran

Chorev

Sapir

(2000). Component processes in task switching. Cognitive Psychology, 41(3), 211–253. https://doi.org/10.1006/cogp.2000.0736

36.

Mok

Johns

Miller

(2017). Tunneled in: Drivers with active secondary tasks need more time to transition from automation proceedings of the 2017 Acm Sigchi Conference on Human Factors in Computing Systems. Association for Computing Machinery. https://doi.org/10.1145/3025453.3025713

37.

Monsell

(2003). Task switching. Trends in Cognitive Sciences, 7(3), 134–140. https://doi.org/10.1016/s1364-6613(03)00028-7

38.

Naujoks

Purucker

Wiedemann

Marberger

(2019). Noncritical state transitions during conditionally automated driving on German freeways: Effects of non-driving related tasks on takeover time and takeover quality. Human Factors, 61(4), 596–613. https://doi.org/10.1177/0018720818824002

39.

Newton

D. W.

LePine

J. A.

Kim

J. K.

Wellman

Bush

J. T.

(2020). Taking engagement to task: The nature and functioning of task engagement across transitions. Journal of Applied Psychology, 105(1), 1–18. https://doi.org/10.1037/apl0000428

40.

Ouellette

J. A.

Wood

(1998). Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior. Psychological Bulletin, 124(1), 54–74. https://doi.org/10.1037/0033-2909.124.1.54

41.

Pampel

S. M.

Large

D. R.

Burnett

Matthias

Thompson

Skrypchuk

(2019). Getting the driver back into the loop: The quality of manual vehicle control following long and short non-critical transfer-of-control requests: TI: NS. Theoretical Issues in Ergonomics Science, 20(3), 265–283. https://doi.org/10.1080/1463922x.2018.1463412

42.

Reimer

Donmez

Lavallière

Mehler

Coughlin

J. F.

Teasdale

(2013). Impact of age and cognitive demand on lane choice and changing under actual highway conditions. Accident Analysis & Prevention, 52, 125–132. https://doi.org/10.1016/j.aap.2012.12.008.

43.

Reiss

(2004). Multifaceted nature of intrinsic motivation: The theory of 16 basic desires. Review of General Psychology, 8(3), 179–193. https://doi.org/10.1037/1089-2680.8.3.179

44.

Roche

Somieski

Brandenburg

(2019). Behavioral changes to repeated takeovers in highly automated driving: Effects of the takeover-request design and the nondriving-related task modality. Human Factors, 61(5), 839–849. https://doi.org/10.1177/0018720818814963

45.

SAE International (2021). Surface vehicle recommended practice, taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles, J3016. SAE International.

46.

Schindler

Lilienthal

A. J.

(2020). Students’ creative process in mathematics: Insights from eye-tracking-stimulated recall interview on students’ work on multiple solution tasks. International Journal of Science and Mathematics Education, 18(8), 1565–1586. https://doi.org/10.1007/s10763-019-10033-0

47.

Shipstead

Harrison

T. L.

Engle

R. W.

(2016). Working memory capacity and fluid intelligence: Maintenance and disengagement. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 11(6), 771–799. https://doi.org/10.1177/1745691616650647

48.

Sun

Cao

Tang

(2021). Shaping driver-vehicle interaction in autonomous vehicles: How the new in-vehicle systems match the human needs. Applied Ergonomics, 90, 103238. https://doi.org/10.1016/j.apergo.2020.103238.

49.

Tan

Zhang

(2022). A computational cognitive model of driver response time for scheduled freeway exiting takeovers in conditionally automated vehicles. Human Factors, 00187208221143028. https://doi.org/10.1177/00187208221143028.

50.

Tivesten

Dozza

(2015). Driving context influences drivers’ decision to engage in visual-manual phone tasks: Evidence from a naturalistic driving study. Journal of Safety Research, 53, 87–96. https://doi.org/10.1016/j.jsr.2015.03.010.

51.

van der Heiden

R. M. A.

Iqbal

S. T.

Janssen

C. P.

(2017). Priming drivers before handover in semi-autonomous cars. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. https://doi.org/10.1145/3025453.3025507

52.

Wandtner

Schömig

Schmidt

(2018). Secondary task engagement and disengagement in the context of highly automated driving. Transportation Research Part F: Traffic Psychology and Behaviour, 58, 253–263. https://doi.org/10.1016/j.trf.2018.06.001.

53.

Weaver

B. W.

DeLucia

P. R.

(2020). A systematic review and meta-analysis of takeover performance during conditionally automated driving. Human Factors, 64(7), 1227–1260. https://doi.org/10.1177/0018720820976476

54.

Weinstein

(2018). Mind-wandering, how do I measure thee with probes? Let me count the ways. Behavior Research Methods, 50(2), 642–661. https://doi.org/10.3758/s13428-017-0891-9

55.

Wickens

C. D.

(2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2), 159–177. https://doi.org/10.1080/14639220210123806

56.

Worle

Metz

Othersen

Baumann

(2020). Sleep in highly automated driving: Takeover performance after waking up. Accident Analysis & Prevention, 144(9), 105617. https://doi.org/10.1016/j.aap.2020.105617

57.

Kihara

Takeda

Sato

Akamatsu

Kitazaki

(2019). Effects of scheduled manual driving on drowsiness and response to take over request: A simulator study towards understanding drivers in automated driving. Accident Analysis & Prevention, 124, 202–209. https://doi.org/10.1016/j.aap.2019.01.013.

58.

Kihara

Takeda

Sato

Akamatsu

Kitazaki

Nakagawa

Yamada

Oka

Kameyama

(2021). Eye movements predict driver reaction time to takeover request in automated driving: A real-vehicle study. Transportation Research Part F: Traffic Psychology and Behaviour, 81, 355–363. https://doi.org/10.1016/j.trf.2021.06.017.

59.

Guo

Wang

(2022). Effect of multiple monitoring requests on vigilance and readiness by measuring eye movement and takeover performance. Transportation Research Part F: Traffic Psychology and Behaviour, 91, 179–190. https://doi.org/10.1016/j.trf.2022.10.001.

60.

Yanko

M. R.

Spalek

T. M.

(2014). Driving with the wandering mind: The effect that mind-wandering has on driving performance. Human Factors, 56(2), 260–269. https://doi.org/10.1177/0018720813495280

61.

Yun

Yang

J. H.

(2020). Multimodal warning design for take-over request in conditionally automated driving. European Transport Research Review, 12(1), 34. https://doi.org/10.1186/s12544-020-00427-5

62.

Zeeb

Buchner

Schrauf

(2015). What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accident Analysis & Prevention, 78, 212–221. https://doi.org/10.1016/j.aap.2015.02.023.

63.

Zhang

Kumada

(2018). Automatic detection of mind wandering in a simulated driving task with behavioral measures. PLoS One, 13(11), Article e0207092. https://doi.org/10.1371/journal.pone.0207092

Exploring the Timing of Disengagement From Nondriving Related Tasks in Scheduled Takeovers With Pre-Alerts: An Analysis of Takeover-Related Measures

Abstract

Objectives

Background

Method

Results

Conclusion

Application

Keywords

Introduction

Scheduled Takeovers and Takeover Requests

NDRT Disengagement

NDRT Disengagement Entails the Cessation of Perceptual, Cognitive, and Responding Activities Related to NDRT Within a Defined Timeframe

The Current Study

Methods

Participants

Apparatus

Experimental Design

Visual and Physical Disengagement Recording

Cognitive Disengagement Recording

Pilot Test for Validating the Method for Recording Cognitive Disengagement

Posttest Interview

Procedures

Dependent Variables

Data Analysis

Results

Identification of NDRT Disengagement Timings

Takeover Performance

Visual Disengagement During the Prewarning Stage

Posttest Interview

Discussion

Identification of NDRT Disengagement Timings

Analysis of Takeover Performance

Visual Disengagement During the Prewarning Stage

Posttest Interview

Implications

Limitations and Future Study

Conclusion

Key Points

Footnotes

Acknowledgments

ORCID iDs

Clues in the Word Puzzle Game

Author Biographies

References