Inferring Hidden Attentional States in Driving: A Bayesian Approach to Modeling Distraction and Secondary Task Engagement

Abstract

Objective

To develop and validate a computational framework that infers individualized attention strategies and latent distraction states to support personalized modeling of multitasking behavior and intervention.

Background

Driver distraction from in-vehicle systems is a growing safety concern. However, the level of distraction is often latent and varies significantly across individuals. Existing models typically overlook these differences, limiting their effective use for personalized interventions.

Method

We introduce a Partially Observable Semi-Markov Decision Process (POSMDP) to model hidden attentional dynamics and attention allocation decisions. Using behavioral data, including glance behavior, velocity, and pupillometry, from a high-fidelity driving simulator with 18 participants, we estimate personalized reward functions that reflect each driver’s subjective valuation of secondary task utility versus safety cost.

Results

The method accurately infers distraction states and recovers participant-specific utility weights governing the trade-off between secondary task benefit and driving cost. Compared to a well-established 2-s glance rule, it improves detection of distraction events and reveals individual variability in attention strategies. Some drivers exhibit highly conservative profiles, while others assign greater value to secondary tasks, even under high distraction. Counterfactual simulations show how perceived task importance could modulate visual attention behavior across individuals.

Conclusion

Our POSMDP-based framework provides an interpretable and individualized account of driver attention allocation, capturing both latent states and behavioral variability.

Application

This model enables the development of personalized, risk-sensitive driver assistance systems that adapt to individual attention strategies, enhancing road safety through context-aware, graded interventions.

Introduction

Despite advances in vehicle automation, human drivers are expected to continue playing a key supervisory role in automated systems for the foreseeable future (Kashevnik et al., 2021). The growing integration of technology in in-vehicle information systems (IVIS) has raised concerns about driver distraction, which is increasingly recognized as a major contributor to vehicle accidents and fatalities (Zhong et al., 2024). According to multiple resource theory (Wickens, 2008), driving and secondary (non-driving related) tasks compete for limited cognitive resources across visual, auditory, manual, and cognitive domains. Distraction increases when both tasks draw from the same resource pool, such as visual input for driving and screen-based IVIS interactions. In Salvucci and Taatgen (2008), the authors conceptualize multitasking as threads of cognitive activity that compete for bottleneck resources (e.g., declarative memory and procedural execution). This theory provides an account of how drivers are able to manage multiple tasks at the expense of some performance degradation.

In this context, understanding the driver’s internal attentional state, whether fully engaged with the driving task or partially distracted, is critical for evaluating its impact on driving performance and safety. We specifically focus on visual–manual distraction arising from interactions with IVIS, rather than auditory or purely cognitive distraction. Studies have shown that drivers differ in how they choose to engage with IVIS. For example, some drivers adopt a cautious approach, while others interact with systems impulsively or without much deliberation (Lerner et al., 2008). Sometimes, drivers may even prioritize secondary tasks during driving, particularly when these tasks (e.g., responding to a work email) seem more engaging or time-sensitive (Lee, 2014). These behavioral differences raise important questions for human factors research: How do drivers assess the cost or risk of engaging in secondary tasks while driving? And do drivers with different decision-making styles exhibit distinct patterns of distraction or task-switching behavior? Quantifying driver decisions can help develop personalized and targeted distraction mitigation strategies, such as optimizing user interface design, tailoring public education and driver training programs, and determining appropriate criteria for warnings (Lerner et al., 2008).

Answering these questions requires accurately measuring distraction; however, this is inherently difficult due to the latent and context-dependent nature of attentional states during driving. For instance, not all off-road glances are distracting, mirror checks or speedometer glances are necessary for safe driving and should not be misclassified as distraction (Schmitt et al., 2016). When multitasking, drivers often fail to recognize their attentional state due to reduced self-monitoring, hindering both real-time adjustments and learning about their multitasking limits (Sanbonmatsu et al., 2016; Starkey & Charlton, 2020; Watson & Strayer, 2010). These observations highlight that drivers may not be fully aware of their distraction state. Instead, drivers rely on their own beliefs about their attentional state, inferred from indirect cues such as changes in driving performance, to guide their decisions regarding engagement in secondary tasks.

In this paper we develop a Bayesian model of subjective perception of hidden/latent (or partially observable) attentional states that evolve according to a controlled Semi-Markov process. The proposed model captures how drivers allocate attention and manage multitasking under varying levels of distraction. Using noisy behavioral signals, including glance behavior, vehicle dynamics, and pupillometry, we develop a two-stage structural estimation procedure to infer latent distraction levels and driver-specific decision policies, and to estimate how perceived rewards associated with secondary task engagement evolve as distraction increases. To test the model, we designed a driving simulator-based experiment involving human participants to observe interactions with IVIS under controlled task conditions. With the experiment data, we verify that the proposed model (i) provides a quantitative account of how latent attentional states evolve and (ii) exhibits improved detection performance when compared with the widely accepted baseline. More importantly, the proposed model helps to understand individual differences by estimating driver-specific utility weights that reflect how each participant weighs the benefits of secondary task engagement against the cost to driving safety. Finally, to confirm that our findings from participant-level estimation are robust, we also estimated a Hierarchical Bayesian version of the model. This analysis corroborates the main results, confirming the same systematic trade-offs and behavioral patterns at the population level. These utility profiles enable simulation of behavioral sensitivity, demonstrating that increases in perceived secondary task importance lead to prolonged off-road glances. Given that individual differences in multitasking strategies are well-established (Jokinen et al., 2021) and that driver assistance technologies increasingly emphasize personalization (Yi et al., 2019), our approach offers a framework for capturing individualized patterns of attention allocation and informing targeted, graded interventions based on driver profiles.

This paper is structured as follows. We begin by reviewing related computational models of driver distraction. Next, we introduce the architecture and computational principles of our proposed model. We then detail the experimental design and data collection procedures used to empirically validate the model with human participants in a simulated driving environment. Finally, we demonstrate the model’s capability to assess behavioral sensitivity through simulations exploring how drivers adjust attention allocation in response to changing task demands and perceived task importance.

Background and Related Work

Driver Distraction

The Task-Capability-Interface (TCI) model (Fuller, 2005) and Intelligent Driver Model (IDM) (Treiber et al., 2000) are widely used in driver distraction research. TCI examines the dynamics between the driving demands and driver capabilities, while IDM models the effects of distractions on car-following and traffic flow. These models are often combined to study driver distraction (Hoogendoorn, Van Arem, & Hoogendoorn, 2013; Saifuzzaman et al., 2015). Another school of driver models use task-independent cognitive architectures to model human perception, cognition, and motor responses. ACT-R-based models, such as Distract-R (Salvucci, 2005), predict distraction effects, while the Queueing Network–Model Human Processor (QN-MHP) quantifies cognitive workload and visual attention allocation (Feng et al., 2017). These models assess distraction impacts on perceptual-motor (Dario D. Salvucci, 2001) and cognitive tasks (Salvucci, 2019) but do not provide a structural account of how attentional states vary over time.

Several data-driven approaches based on machine learning (ML) have been employed to predict driver visual-manual distraction. Common algorithms include supervised learning models, such as support vector machines (Ersal et al., 2010; Liang et al., 2007; Li et al., 2017), decision trees (McDonald et al., 2020; Zhang et al., 2004), hidden Markov models (Kanaan et al., 2019), Bayesian networks (Liang & Lee, 2014), and deep-learning architectures (Huang et al., 2021; Masood et al., 2020; Ou & Karray, 2019; Tran et al., 2018; Wollmer et al., 2011; Xing et al., 2019). While effective, they do not formally model decision-making processes in driving scenarios where uncertainty arises from limited attention. This is a crucial consideration, especially when multiple tasks vie for the same cognitive resources. In what follows, we discuss the frameworks to study such decision-making processes.

Partially Observable Markov Decision Processes

The Partially Observable Markov Decision Process (POMDP) framework (e.g., Krishnamurthy, 2016) models a forward-looking utility-maximizing agent operating in an environment with unobservable states, where the evolution of these states follows a hidden Markov model that is determined (or controlled) by the agent’s actions. Because the state is not directly observable, the agent maintains a belief, a probability distribution over hidden state space, that serves as a sufficient statistic for decision-making under uncertainty. With observed data in the form of sequences of observations and actions, the structure of the POMDP (i.e., reward function, state transition probabilities, and observation probabilities) can be estimated (Chang et al., 2023). POMDPs have been applied to infer driver perception during lane-keeping with secondary tasks (Schmitt et al., 2016), model multitasking behavior in hierarchical reinforcement learning frameworks (Jokinen et al., 2021), and support AV-related challenges such as driver attentiveness during perception hand-offs (Pant et al., 2022), motion planning, and intent prediction (Sadigh et al., 2016; Li, Zhao, & Wang, 2022; Danesh et al., 2023). More recently, active inference formulations of POMDPs have been introduced to model adaptive human driving and automated vehicle takeovers, demonstrating how goal-directed and uncertainty-resolving behaviors emerge from the principle of expected free energy minimization (Engström et al., 2024; Wei et al., 2025). While a POMDP model accounts for unobserved states that affect decision making, it only allows for memoryless dynamics: that is, the distribution of the sojourn time (the number of consecutive time steps spent in a state before transitioning to another) is independent of the time spent in that state. However, attentional state dynamics in a driving task are not memoryless. For instance, driver inattention persists for varying durations influenced by secondary task demands and external stimuli (Arvin & Khattak, 2020). In this paper, we use a semi-Markov model to realistically represent driver behavior, where semi-Markov state dynamics allows general state sojourn time distributions. The semi-Markov model has been used to describe latent human psychological state dynamics to model mental fatigue (Wang et al., 2022), to describe driver response time to alerts (Hwang et al., 2020) or waiting periods when interacting with pedestrians (Zhang & Fricker, 2020). Given these advantages, we incorporate semi-Markov unobserved state dynamics into POMDP framework, resulting in a so-called Partially Observable Semi-Markov Decision Process (POSMDP) to better capture the temporal variability of driver attentional states and decision making in dual-task scenarios.

A Model of Distraction and Attention Dynamics

Standing Assumptions

To capture key behavioral regimes, we simplify the hidden driver state of attention to low distraction ( $s_t=0$ ) and high distraction ( $s_t=1$ ). At each decision epoch $t>0$ , the driver’s attentional state $s_t\in S:=\left\{0,1\right\}$ and the sojourn time ${\tau }_t$ (the remaining time in state $s_t$ ) are unobservable to both the agent or external observers. Due to limited self-monitoring while multitasking, drivers often fail to recognize their own attentional state, reducing their ability to make immediate compensatory adjustments or leading to misjudgments about their capacity to multitask safely (Sanbonmatsu et al., 2016). However, a noisy observation $z_t\in Z\subset {\mathbb{R}}^m$ (e.g., a measure of driving performance and cognitive workload) that correlates with the unobserved attentional state is used to infer the attentional state. Thus, for example, drivers engaged in secondary tasks could exhibit dangerous or erratic driving behaviors (Basacik et al., 2011; Caird et al., 2014; Choudhary & Velaga, 2017b; World Health Organization, 2011) inducing a posterior with high likelihood for distracted state. For this study, two key observations are utilized. First, velocity $\left(z_v\right)$ serves as a significant indicator of distraction, as distracted drivers tend to reduce their speed to maintain safety (Ortiz-Peregrina et al., 2020; Papantoniou, Papadimitriou, & Yannis, 2017). Second, our model utilizes pupillometry $\left(z_p\right)$ , specifically the percentage change in pupil size (PCPS), to measure workload, as pupil diameter usually increases with cognitive load in tasks (Kraus et al., 2023; Lohani et al., 2019; Shahini et al., 2024).

The action $a_t\in A:=\left\{0,1,2\right\}$ represents the driver’s visual attention allocation where $a_t=0$ corresponds to gaze maintained on the roadway, including fixation and brief saccades shorter than 0.1 s, thereby capturing the predominant on-road viewing behavior while driving; $a_t=1$ stands for off-road glances between 0.1 s and 2 s; and $a_t=2$ stands for off-road glances exceeding 2 s, which pose the highest risk (Klauer et al., 2006; NHTSA, 2012). Visual attention is commonly assessed by examining glance behavior and the duration of off-road glances (those not in the direction of travel) (Birrell & Fowkes, 2014; Risteska et al., 2021).

We make three standing assumptions (formal definitions provided in Appendix A):

Assumption 1

(Conditional Independence): Given the current state, duration, and action, the next attentional state, duration, and observations are conditionally independent.

Assumption 2

(Sojourn time distribution): The time a driver remains in a given attentional state follows a binomial distribution, capturing discrete durations consistent with our experimental sampling.

Assumption 3

(Observation probabilities): Observable variables (velocity and PCPS) are assumed to follow a state-dependent multivariate normal distribution. For tractability, we assume independence and set the covariance terms to zero because we examined the empirical correlation between velocity and PCPS using the Pearson correlation coefficient and found a statistically significant but weak negative correlation ( $\rho =-0.24$ , $p<0.001$ ).

Thus, the intertemporal transition of both the attentional state and its associated duration is governed by transition probabilities, expressed as ${\mathbb{P}}_{{\theta }_D}\left(s_{t+1},{\tau }_{t+1},z_{t+1}|s_t,{\tau }_t,a_t\right)$ where the parameter ${\theta }_D=\left\{{\theta }_1,{\theta }_2,{\theta }_3\right\}$ . Here, ${\theta }_1$ describes the state transition probability, ${\theta }_2$ represents the sojourn time distribution, and ${\theta }_3$ is the observation probability.

A two-state model (low distraction vs. high distraction) was adopted for both theoretical and empirical reasons. Conceptually, the binary representation aligns with how visual–manual distraction is characterized in the human factors literature, as transitions between attentive and distracted modes of control (He et al., 2023; Jokinen et al., 2021). Experimentally, the secondary task (map search) was designed to induce distinct episodes of distraction rather than graded engagement levels, making a two-state formulation consistent with the task structure. Moreover, preliminary analyses with a three-state variant produced an additional state reflecting minor fluctuations in driving behavior, but this state was not stable or interpretable across participants. For parsimony and interpretability, we therefore retained the two-state model.

Model of Distracted Driver: Bayesian Inference

To model how a driver interprets incoming information and updates their understanding of their attentional state, we adopt a Bayesian framework. In this view, the driver is treated as a Bayesian agent who maintains an internal belief about whether they are focused or distracted, based on the history of behavior and observations. Let ${\zeta }_t:=\left\{a_0,z_1,a_1,...a_{t-1},z_t\right\}$ , denote the observed history of actions and observations until time $t$ . Given an initial prior distribution $x_0$ , the driver’s belief (or posterior distribution) $x_t\triangleq \left\{\mathbb{P}\left(s_t,{\tau }_t|{\zeta }_t,x_0\right):s_t\in S,{\tau }_t\in {\mathbb{N}}^{+}\right\}$ over attentional states $s_t$ and the corresponding sojourn time ${\tau }_t$ can be computed. The belief can be updated recursively as follows:

$x_{t+1}\left(s_{t+1},{\tau }_{t+1}\right)=\frac{{\displaystyle \sum }_{s_t,{\tau }_t}P\left(s_{t+1},{\tau }_{t+1},z_{t+1}|s_t,{\tau }_t,a_t\right)x_t\left(s_t,{\tau }_t\right)}{\sigma \left(x_t,a_t,z_{t+1}\right)},$

(1)

where

$\sigma \left(x_t,a_t,z_{t+1}\right)={\displaystyle \sum }_{s_{t+1},{\tau }_{t+1}}{\displaystyle \sum }_{s_t,{\tau }_t}P\left(s_{t+1},{\tau }_{t+1},z_{t+1}|s_t,{\tau }_t,a_t\right)x_t\left(s_t,{\tau }_t\right).$

(2)

Intuitively, equation (1) describes how the driver integrates past information with the most recent observation to update their internal belief of attentional focus. For instance, after a brief off-road glance ( $a_t$ ), the driver’s inferred attentional state ( $s_{t+1}$ ) depends not only on that glance but also on their prior level of distraction and how long they have been in that state ( ${\tau }_t$ ).

Model of Distracted Driver: Attention Reward vs Cost Tradeoffs

After updating their belief about the current attentional state, the driver selects an action based on that belief, deciding how much attention to allocate to the primary driving task versus the concurrent secondary task. When a secondary task appears (e.g., visual search prompt on the in-vehicle system), the driver must divide attention between the primary driving task (e.g., maintaining a safe following distance) and the secondary task. Given that visual attention has a limited capacity (Reimer & Schubert, 2020), the driver is modeled as a rational agent aiming to maximize expected reward by balancing the potential benefits of completing the secondary task against the cost and risks of compromised driving performance (see Jokinen et al., 2021). We thus define the one-step reward $r_{{\theta }_R}\left(s,a\right)$ , which describes the agent’s preferences over state-action pairs $\left(s,a\right)\in S\times A$ as:

$r\left(s,a\right)=w\left(s\right)B\left(a\right)+\left(1-w\left(s\right)\right)\left(-C\left(a\right)\right)$

(3)

where $B\left(a\right)$ represents the benefit or utility of completing the secondary task, $-C\left(a\right)$ captures the cost associated with attentional diversion from the primary driving task, and $w\left(s\right)\in \left[0,1\right]$ is a state-dependent weight that governs the driver’s trade-off between benefit and cost.

Model of Distracted Driver: Attention Allocation Policies

Based on the driver’s preferences captured by the reward function $r\left(s,a\right)$ and the updated belief $x_t$ , attention allocation is modeled through a probabilistic decision rule, or policy, that specifies how likely the driver is to choose each level of attentional effort. To model this attention allocation process, we use a randomized visual attention policy, $\pi \left(a|x\right)$ ; it is the probability of choosing attention effort $a\in \left\{0,1\right\}$ given the Bayesian belief distribution $x$ . Specifically, we posit the following reward-maximizing attention allocation model:

$\pi \left(·|x\right):=\arg \underset{\widehat{\pi }}{\max }\left\{{\mathbb{E}}_{s_t\sim x_t,a_t\sim \widehat{\pi }\left(·|x_t\right)}\left[{\displaystyle \sum }_{t\ge 0}{\beta }^t\left(r\left(s_t,a_t\right)+{ϵ}_t\right)|x_0=x\right]\right\}$

(4)

where $\beta \in [0,1)$ is a discount factor and ${\left\{{ϵ}_t\right\}}_{t\ge 0}$ is an independent and identically distributed sequence of reward perturbations. The model described in equation (4) can be seen as a dynamic version of discrete choice models used to model human behavior (see, e.g., Ben-Akiva & Lerman, 1985 for applications in transportation). According to the model, given state $x$ , the driver chooses an action that is drawn from a policy that maximizes their expected total discounted reward, taking into account both immediate and future consequences. The discount factor $\beta \in [0,1)$ reflects how much the driver values future outcomes relative to immediate ones. The random variable ${ϵ}_t$ models utility perturbation accounting for variation in observed actions for the same state. If the distribution of the perturbation ${ϵ}_t$ follows a standard Gumbel distribution, the optimal policy takes on a softmax form (see, e.g., Chang et al., 2023) given by:

$\pi \left(a_t|x_t\right)=\frac{\exp \hspace{0.17em}Q\left(x_t,a_t\right)}{{\displaystyle \sum }_{a^{\prime }}\exp \hspace{0.17em}Q\left(x_t,a^{\prime }\right)},$

(5)

where the state-action value function $Q\left(x_t,a_t\right)$ can be written as

$Q\left(x_t,a_t\right)=r\left(x_t,a_t\right)+\beta {\displaystyle \sum }_{z_{t+1}}\sigma \left(x_t,a_t,z_{t+1}\right)V\left(x_{t+1}\right)$

and $V\left(x_{t+1}\right):=\log {\displaystyle \sum }_a\exp \hspace{0.17em}Q\left(x_{t+1},a\right)$ is the value function. Here, $Q\left(x_t,a_t\right)$ represents the expected value of choosing action $a_t$ given belief state $x_t$ at time $t$ , assuming optimal decisions thereafter.

To summarize, the driver chooses an action based on the decision policy in equation (5), which maximizes the expected total discounted reward for allocating visual attention resources given the competing demands of driving safely and managing other tasks or information sources.

Estimation

We estimated the model parameters using maximum likelihood estimation (MLE). Specifically, the goal was to determine the parameter sets governing the dynamics of driver distraction, ${\theta }_D$ , and the reward function, ${\theta }_R$ , that jointly maximize the likelihood of the observed data. Given $N$ driving sequences, each consisting of $T$ time steps, the log-likelihood to estimate $\theta =\left\{{\theta }_D,{\theta }_R\right\}$ can be written as

$\begin{array}{ll}\log \hspace{0.17em}l\left(\theta \right)& ={\displaystyle \sum }_{i=1}^N{\displaystyle \sum }_{t=0}^T\left[\log \hspace{0.17em}{\sigma }_{{\theta }_D}\left(x_{t,i},a_{t,i},z_{t+1,i}\right)+\log \hspace{0.17em}{\pi }_{\theta }\left(a_{t,i}|x_{t,i}\right)\right].\end{array}$

(6)

The detailed likelihood formulation is provided in Appendix B. The estimation is performed in two stages. In the first stage, we estimated the parameters ${\theta }_D$ associated with the driver’s state dynamics and observation process by maximizing the likelihood of observed cues given the current belief and action. In the second stage, we used the estimated ${\widehat{\theta }}_D$ to infer ${\theta }_R$ , the parameters of the reward function, by maximizing the likelihood of observed actions given the belief state. The model is evaluated using data from a controlled experiment with human participants, described in the following section.

Experiment Design

Experimental Apparatus and Environment

Driving Simulator

The study utilized a high-fidelity driving simulator (Figure 1) by Realtime Technologies, Inc. The simulator featured a 300° field of view, a four camera video capturing system and driving behavior data was captured at a sampling rate of 60 Hz. SimCreator DX software was employed to design the scenarios. To simulate interactions with IVIS, participants were required to perform non-driving related tasks on an iPad. Specifically, the Google Maps application was used to replicate typical IVIS interactions and potential driver distractions.

Figure 1.

The experimental apparatus and environment.

Eye Tracking System

To collect driver pupil data, the study employed a Pupil-core eye tracking system manufactured by Pupil Labs, Germany (Figure 1). This system comprised three key components: a world camera and two eye cameras. The eye cameras utilized 3-D models to detect and track pupil movements with high precision. Calibration of the pupil tracking was achieved using Apriltag markers (Figure 2). These markers were positioned on the central driving screen, enabling the system to distinguish between on-path and off-path glances.

Figure 2.

Experimental setup showcasing construction zone and eye-tracking markers.

Environment Design

The experimental scenario featured a four-lane highway, with two lanes allocated for each direction of travel. The total simulated driving distance spanned approximately 15 miles and approximately 12 min to complete. Ambient vehicles were added in the inner lane to simulate realistic traffic conditions. Traffic density was set to represent stable flow conditions, where drivers are influenced by the presence of other vehicles, but desired speeds remain unaffected.

To enhance engagement and realism, the route incorporated a construction zone setting for 15% of the route (Figure 2). The route also included two curved sections to reflect typical highway geometry.

Primary Driving Task

The study’s primary task was designed to simulate realistic highway driving conditions, focusing on vehicle following behavior. Participants were instructed to follow a lead vehicle traveling at 70 mph in the outer lane, while maintaining a safe relative distance throughout the experiment (Figure 1). An auditory alert system ensured adherence to a 1.75 s headway threshold ( $\sim$ 60 m), set based on research indicating that at higher velocities, drivers tend to perceive time headway below 1.5 s as uncomfortable (Siebert et al., 2014). The simulator continuously monitored the time headway, and whenever it exceeded 1.75 s, a brief auditory tone was automatically played to prompt participants to accelerate and close the gap. Reminders were separated by at least one second to ensure that consecutive alerts did not overlap.

Secondary Task: Simulated In-Vehicle System Interaction

To simulate in-vehicle distractions, participants performed intermittent map search tasks requiring extended glances away from the road. These tasks, designed to require high-attention and be cluttered (Pankok & Kaber, 2018), involved both listening to routine navigation questions and interacting with the system to find answers (Figure 3). Questions included:

(1) “Can you find the gas station with the lowest gas price?”

(2) “Can you find the CVS pharmacy that is open now?”

(3) “Can you find the closest gas station?” (defined as the gas station with the shortest time deviation from the current route, shown in minutes in Figure 3(b)).

Figure 3.

Secondary task interface for map search tasks.

The procedure involved three steps (see Figure 4): first, participants clicked on the search icon to access options (Step 1). Options include categories such as gas stations, pharmacies, and restaurants, similar to those in common navigation apps. Next, they selected the relevant category, either “gas station” for questions 1 and 3 or “pharmacy” for question 2 (Step 2). Finally, they clicked on the correct choice from the available options to complete the task (Step 3). The IVIS task was limited to the three questions listed above. These same three questions were used in both practice and experimental drives, with randomized order within each drive. IVIS content was updated dynamically so the correct answer for each question varied across drives and across repeated presentations of the same question. Secondary tasks were presented randomly, with a minimum interval of 60 s between tasks. Studies have shown that the effects of distraction can persist for 10–40 s after the distracting task ends (Bowden et al., 2019; Strayer et al., 2016). Our 60-s interval was chosen conservatively to ensure adequate recovery time across various individual differences and distraction types. To introduce variability in when tasks appeared, we added a random delay (0–5 s) to the 60-s base interval.

Figure 4.

Secondary task operations: (1) Participants click on the search icon, (2) they select the relevant category, either “gas station” for questions 1, 3 or “pharmacy” for question 2, and (3) they click on the correct choice from the options provided to complete the task. Step 3 varies based on the specific question being asked, as illustrated in Figure 3.

Based on trials of our secondary task setup, we found the average task completion time was 12 s. To standardize the experiment, we set a 15-s time limit per task, measured from the question onset. An auditory notification marked the end of the task, prompting participants to disengage from the IVIS and refocus on driving.

Procedure

The experiment began with participants completing an informed consent form. A 2-min baseline pupil size measurement was collected using an eye tracker. Participants then trained on the simulator and completed a trial run of the secondary task. The main experiment included four 12-min driving sessions, with 5–10-min breaks to reduce fatigue. Participants were instructed to treat the scenario as real-world driving, with no prioritization guidance for driving or secondary tasks. After the sessions, a final 2-min baseline measurement was taken, followed by a questionnaire collecting demographics. The average of the two baselines was used to calculate percent changes in pupil size during driving tasks (Zahabi et al., 2023).

Participants

We collected repeated driving sessions from each participant, providing the detailed participant-specific trajectories needed for our computational model and allowing us to recover participant-specific behavioral profiles. This focus aligns with recent POMDP computational modeling frameworks that rely on detailed individual trajectories and modest sample sizes (Jokinen et al., 2021; Schmitt et al., 2016). We also conducted an a priori power analysis in G*Power software (Faul et al., 2007) to determine the required sample size, using $\alpha =0.05$ , power $1-\beta =0.95$ and an effect size estimated from pilot test data. To remain conservative, we used the smallest effect size observed across the dependent variables (velocity and PCPS). This analysis indicated that a minimum of 14 participants is required to detect effects of that magnitude in paired comparisons. Therefore, eighteen (18) male participants (ages 20–33 years, M = 25, SD = 4.1 years) were recruited from the College Station, TX area for this study. All participants held a valid driver’s license at the time of the experiment and had 20/20 vision. As compensation for their time, participants received $25 per hour. The study was approved by the Institutional Review Board and adhered to ethical guidelines.

Estimation Results

Data Pre-Processing

Each driving session lasted 12 min and was segmented into three trajectories, using road curves as natural boundaries. Data was not collected during curves to avoid curvature-induced driving variations (Rosas-López et al., 2021; Xu et al., 2022). Data from all sources were synchronized using timestamps and smoothed with a 1-s rolling average. Eye-tracking data with confidence below 95% were filtered out, and trajectories with over 20% missing or corrupted data were excluded from analysis. For model inference, a trajectory corresponded to one straight road segment and consisted of a continuous sequence of 1-s decision epochs, where each epoch contained the driver’s observed variables and actions. On average, for each participant, each trajectory consists of around 250 discrete decision epochs.

We examined participants’ driving and secondary-task performance to provide behavioral context for the subsequent modeling analyses. During baseline (non-IVIS) driving, participants maintained stable car-following performance, with lane offset exceeding the 0.9 m threshold in approximately 5% of samples and time headway exceeding the 1.75 s threshold in 3% of samples on average. No near-crash events were observed, indicating safe and consistent vehicle control. When engaged in the IVIS task, performance degraded moderately, lane-offset exceedances increased to about 16% on average (range: 1–41%), reflecting individual variability in lateral control during distraction. Nevertheless, participants achieved an average IVIS task success rate of 82%, suggesting that they could generally manage both tasks, albeit with measurable costs to vehicle control.

To verify that the observed heterogeneity in our estimates reflects true individual differences rather than estimation noise, we employed two complementary estimation approaches. First, we estimated participant-specific parameters by fitting the POSMDP model independently to each driver’s data. Second, to assess the robustness of these patterns, we implemented a hierarchical Bayesian extension that pools information across drivers to jointly estimate group-level and participant-level parameters (see Appendix E for details). In the sections that follow, we present the results from both approaches side-by-side to demonstrate that the core behavioral structure persists even after hierarchical shrinkage.

Attention Dynamics

In general, partially observable Semi-Markov Decision Processes (POSMDPs) allow the process to transition from a hidden state back to itself at specific transition epochs, that is, when $\tau =0$ (Cao, 2003; Khodadadi et al., 2014; Zhang & Revie, 2016). This formulation captures the possibility that, after a sojourn period, the system may either remain in the same latent state or transition to another, depending on the current state and action. We allow this flexibility to model realistic behavioral persistence. The detailed state transition probabilities for each participant are presented in Table C1 (Appendix C), while Figure 5(a) summarizes the distributions of estimated transition probabilities across participants for each combination of action and state transition. Figure 5(b) presents the corresponding transition distributions estimated under the hierarchical version of the model. Across both modeling approaches, the same core structure of driver attentional dynamics emerges.

Figure 5.

Summary of estimated state transition probabilities across participants.

On-Road Glances and Brief Saccades ( $a=0$ )

Overall, the transitions associated with $a=0$ vary considerably across participants. This implies that on-road glance and brief saccades affect different drivers in various ways (e.g., for some individuals such saccades might lead to distraction while not for others), suggesting diverse baseline dynamics due to individual heterogeneity in learning, perception, or prior experiences. Specifically, (i) when drivers are in a low-distraction state ( $s_t=0$ ) and make brief saccades, there is a moderate probability of maintaining their low-distraction state, indicating that brief saccades are common during low-distraction driving, for example, checking the speedometer or mirrors that are essential for safe driving (Schmitt et al., 2016); (ii) for some participants, even minimal off-road glances increase the likelihood of transitioning from a low-distraction to a high-distraction state (as illustrated in the first-row, first-column panel of Figure 5(a)), suggesting that factors beyond visual attention, such as auditory or cognitive distractions, may play a role (highlights the need for systems that monitor more than just eye movements); and (iii) when drivers are already in a high-distraction state ( $s_t=1$ ) and make on-road glances or brief saccades, most participants show a low probability of remaining in the high-distraction state (as illustrated in the second-row, first-column panel of Figure 5(a)). This suggests that on-road glance and brief saccades may reflect attempts to re-engage with the driving task. The hierarchical model reproduces this overall pattern. While the hierarchical approach smooths some of the participant-specific extremes, the qualitative dynamics remain consistent. In both Figure 5(a) and (b), the transition probabilities under $a=0$ show moderate tendencies toward high-distraction from the low-distraction state and low persistence in the high-distraction state. The detailed group-level estimates underlying these results are provided in the Appendix E.

Short Off-Road Glances ( $a=1$ )

The transitions associated with $a=1$ (short off-road glances) show less variability compared to $a=0$ , with some common trends emerging across participants. For most participants, there is an increased probability of transitioning to a high-distraction state, regardless of their initial state (as illustrated in the second-column of Figure 5). This suggests that even short off-road glances can negatively impact driving performance (Liang et al., 2012). However, participants 2, 12, and 17 deviate from this pattern, exhibiting a higher likelihood of maintaining a low-distraction state when making short off-road glances. The hierarchical model captures the same dominant pattern of increased transitions into the high-distraction state, while reducing the influence of extreme individual estimates. This smoothing is particularly noticeable for the transition $P\left(s^{\prime }=1\mid s=1,a=1\right)$ (as seen in the second row, second column of each panel), where the individual-level distribution is diffuse but the hierarchical estimate produces a more interpretable group-level tendency.

Long Off-Road Glances ( $a=2$ )

The transitions associated with $a=2$ (longer off-road glances) exhibit extreme values (close to 1) of the probability of transitioning to distracted state $s=1$ , indicating strong deterministic effects of these actions (as illustrated in the third-column of Figure 5). The results suggest that prolonged off-road glances ( $>2$ seconds), almost inevitably lead to distraction, consistent with previous findings (Klauer et al., 2006). This highlights the need for intervention strategies aimed at preventing extended off-road glances. The hierarchical version mirrors this effect with minimal variability across participants. This strong alignment across approaches confirms that this behavioral regularity is robust.

Observation Distributions

Figure 6(a) visualizes the difference in the estimated mean velocity between the low-distraction and high-distraction states. The mean velocity for $s_t=0$ (low-distraction driving) was found to be slightly yet consistently higher (e.g., 33.71 m/s for participant 1) compared to the mean velocity of $s_t=1$ (high-distraction driving) (32.42 m/s for participant 1). This difference in velocity suggests a self-regulating tendency among drivers (Bamney et al., 2022; Horberry et al., 2006). When engaged in secondary tasks, drivers appear to compensate for the increased cognitive load by slightly reducing their speed, potentially as a subconscious safety measure (Oviedo-Trespalacios et al., 2017). However, for some participants (subjects 3, 8, 18), we observe a very mild increase in velocity under distracted conditions. Despite these exceptions, the majority of the evidence indicates a stronger tendency to decrease speed under distraction (Emily Parcell et al., 2021), a phenomenon we also observe in most cases, as 15 out of 18 participants exhibited a decrease in velocity when distracted. The hierarchical Bayesian version of the POSMDP model yields a highly similar pattern (Figure 6(b)). Although the hierarchical model introduces small shifts due to the population-level priors, participants who showed a mild increase in velocity under distraction in the individual fits (participants 3, 8, 18) continue to exhibit this pattern.

Figure 6.

Estimated mean velocity for each participant under the low- and high-distraction latent states.

Figure 7(a) illustrates the difference in estimated mean PCPS between the high-distraction and low-distraction states. The low-distraction state ( $s_t=0$ ) exhibited a lower mean PCPS compared to the high-distraction state ( $s_t=1$ ) across all participants. This higher PCPS value in $s_t=1$ correlates with an increased cognitive workload or demand (Biondi et al., 2023), which is consistent with the nature of the secondary task. Such noisy, variable observations highlight the need for a partially observable model, which infers distraction states that direct observation alone cannot reliably capture. Table C2 in the Appendix presents the detailed estimated observation probabilities. The hierarchical Bayesian estimates (Figure 7(b)) show the same structure: the mean PCPS remains consistently higher in the high-distraction state than in the low-distraction state, mirroring the pattern seen in the individually fitted model.

Figure 7.

Estimated mean PCPS for each participant under the low- and high-distraction latent states.

Sojourn Time Distributions

The sojourn time estimates are visualized in Figure 8(a). For the low-distraction state, we have $P_{{\theta }_2}\left({\tau }_{t+1}|s_{t+1}=0,{\tau }_t=0\right)\sim B\left({\theta }_{2,0},75\right)$ , where $n=75$ includes a mandated minimum duration of uninterrupted attentive driving 60 s plus an additional period of 15 s to account for random perturbations. The estimated parameter ${\theta }_{2,0}$ varies between 0.63 (47 s) and 0.88 (66 s), with an average of 0.782 (58.67 s) across participants, aligning closely with the expected attentive driving duration. Similarly, for the distracted state, we have $P_{{\theta }_2}\left({\tau }_{t+1}|s_{t+1}=1,{\tau }_t=0\right)\sim B\left({\theta }_{2,1},15\right)$ , where $n=15$ corresponds to the average duration of the secondary task as defined by our experimental design. The estimated ${\theta }_{2,1}$ ranges from 0.34 (5.1 s) to 0.84 (12.7 s), with an average of 0.65 (9.75 s), suggesting that participants typically took approximately 9.75 s to conclude the secondary task. Detailed numerical estimates of the sojourn time are provided in Table C2 in the Appendix. The hierarchical estimates for sojourn durations (Figure 8(b)) closely match those obtained from the individual POSMDP model (Figure 8(a)). The average duration in the low-distraction state (54.97 s) remains close to the earlier estimate of 58.67 s, and the corresponding average duration for the high-distraction state (8.25 s) similarly aligns with the previous mean of 9.75 s. As shown in Figure 8(b), the estimates from hierarchical method exhibit tighter distributions, reflecting reduced variability when information is shared across participants. While the hierarchical method estimates an informed population-level prior for all parameters across participants, both approaches ultimately recover the same qualitative pattern of attentional dynamics.

Figure 8.

Estimated sojourn time parameters ( ${\theta }_{2,s}$ ) for the low-distraction ( $s=0$ ) and high-distraction ( $s=1$ ) states across participants.

Perceived Utility of Attention Allocation

In our framework, drivers are assumed to make attention allocation decisions based on perceived utilities and costs associated with various actions. As formalized earlier (equation (3)), the reward function denotes the trade-off between the benefit $B\left(a\right)$ and the cost $C\left(a\right)$ associated with each action. To ensure identifiability of the weight parameter $w\left(s\right)$ , we fix $B\left(a\right)$ and $C\left(a\right)$ using post-experiment self-reports. Specifically, participants rated their perception of risk and benefit associated with each glance behavior (action) on a seven-point Likert scale. The questions were as follows. (1) Perceived Risk (Cost): “How likely do you think performing this action (e.g., taking longer off-road glance) could contribute to a dangerous situation or safety-related consequence while driving?” (2) Perceived Benefit: “How useful or important was this action for successfully completing the secondary task shown on the in-vehicle system?”

Responses were averaged across participants to yield empirical proxies for $B\left(a\right)$ and $C\left(a\right)$ . As expected, longer off-road glances were associated with both higher benefit and higher cost. The resulting values are shown in Table 1.

Table 1.

Subjective Ratings of Cost and Benefit for Different Off-Road Glance Durations.

Action $a$	Glance Type	Benefit $B\left(a\right)$	Cost $C\left(a\right)$
0	On-road and brief saccade ( $<$ 0.1 s)	0.0	0.0
1	Short off-road glance (0.1 s–2 s)	4.1	3.0
2	Long off-road glance ( $>$ 2 s)	5.3	6.2

For attention-sensitive tasks, such as reading a long message, entering navigation information, or interpreting a complex visual display, the benefit of sustained attention is high (Strayer et al., 2019), making the relative benefit structure, $B\left(2\right)>B\left(1\right)>B\left(0\right)$ , both intuitive and necessary. The inequality $C\left(1\right)<B\left(1\right)$ implies that for moderate off-road glances, drivers perceive the task benefit to outweigh the risk. This suggests that drivers may be willing to engage in such glances as a low-risk compromise between maintaining driving safety and accomplishing in-vehicle tasks. Further, $C\left(2\right)>B\left(2\right)$ , suggesting that although participants recognize the utility of sustained attention (extended glances) for completing the task, they also perceive it as increasingly risky. These subjective reports are specific to the secondary task and driving context used in our study; in real-world scenarios, these values may vary depending on environmental conditions and task complexity. Table D1 (in Appendix) presents the estimated values of the state-dependent weighting parameter for each participant. These weights capture how strongly each participant prioritizes the benefit of the secondary task relative to its cost when making attention allocation decisions.

Figure 9(a) summarizes the central empirical finding of drivers’ attention strategies. Drivers perceive greater utility in off-road attention when distracted ( $w_1>w_0$ in Figure 9(a)), but the degree of this shift (i.e., $w_1-w_0$ ) varies markedly across individuals. For instance, Participant 5 assigns minimal weight to off-road glances in both low- and high-distraction states, exhibiting a consistently conservative strategy ( $w_0=0.024,w_1=0.176$ ). In contrast, Participant 11 consistently places high value on off-road attention, indicating a relatively greater willingness to engage with the secondary task ( $w_0=0.357,w_1=0.470$ ). A majority of drivers, including Participants 9, 14, and 15, dynamically adjust attention strategy, reallocating visual resources towards secondary tasks. The hierarchical Bayesian model recovers the same directional pattern ( $w_1>w_0$ in Figure 9(b)), mirroring the shift observed in the individual POSMDP fits. This agreement across modeling approaches reinforces the conclusion that once disengagement occurs, drivers assign greater value to secondary-task engagement, reflecting a consistent population-level shift in attentional priorities despite substantial individual variability in magnitude. These findings highlight the heterogeneity in distraction behavior and motivate the development of individualized models of attention allocation.

Figure 9.

State-dependent weights assigned to off-road glances for each participant.

Model Evaluation

Unlike machine learning models that focus on prediction using high-dimensional data, our structural model explains the latent evolution of attentional states, prioritizing interpretability and enabling counterfactual reasoning—features typically lacking in black-box approaches. Given these fundamental differences in goals and design, we do not compare against ML approaches, as, to our knowledge, no existing models offer the same explanatory framework. Instead, we adopt a widely accepted behavioral benchmark, off-road glances exceeding 2 s (Kircher & Ahlström, 2017; Klauer et al., 2006) as a natural and interpretable baseline to evaluate our model.

For evaluation, one trajectory per participant was held out as test data, and the reported results (see Table 2) compile model performance across all participants. For the purpose of validating the inferred states, we defined a proxy for the “ground truth” attentional state based on the onset of the IVIS task. Specifically, we marked the onset of distraction at the presentation of the audio question, which initiates cognitive engagement with the IVIS, and the offset of distraction at the completion of the task, when the participant returns to primary driving. To validate the model, we compare the evolution of the model inferred belief in the high-distraction state against these externally defined episodes that function as a proxy ground truth. Table 2 shows that while the baseline rule detects 68.4% of secondary task events, our model achieves 97.4% detection, indicating an improvement. However, not all distraction events pose the same level of safety risk. To assess safety-critical consequences, we further examine whether the model-predicted distraction periods coincides with lane deviations, a more severe outcome (Choudhary & Velaga, 2017a). Lane offset refers to the lateral distance of the vehicle from the center of the lane. A lane offset threshold of 0.9–1.0 m is used to flag unsafe deviations (Faizan et al., 2018). As shown in the table, our model detects 100% of such events, compared to 70.6% by the baseline.

Table 2.

Comparison of Distraction Detection Performance Between the POSMDP Model and the 2-S Off-Road Glance Baseline, Evaluated on Held-Out Test Data.

Event Type	Total Events	POSMDP Model	Baseline Rule
		(% Detected)	(% Detected)
Secondary task engagements	38	97.4%	68.4%
Lane deviations	17	100%	70.6%

Note. Detection rates are reported for secondary task engagements and lane deviations. “Secondary task engagements” denote IVIS interaction periods, and “lane deviations” indicate cases where IVIS interaction coincided with lateral offsets exceeding 0.9–1.0 m. The POSMDP model consistently outperforms the baseline across both event types.

In addition to detection accuracy, we also estimated the lead time, that is, how much earlier the POSMDP model identified distraction events compared to the baseline 2 s rule (for example, see Figure 10). On average, across all test data, our model detected distraction 1.30 s earlier than the baseline. This advance prediction window is critical for enabling timely interventions and mitigating potential risks. While a shorter baseline threshold (e.g., 1.5 s) could detect distraction earlier, such heuristic rules are fixed. In contrast, our model learns a data-driven representation that adapts to the driver’s behavioral context.

Figure 10.

Illustrative example of lead time estimation.

Counterfactual Simulations for Task Prioritization and Risk Sensitivity

Drivers vary in risk sensitivity, attention allocation strategies, and multitasking capacity, highlighting the need for personalized distraction interventions (Yi et al., 2019). Figure 11 simulates how a driver’s effort allocation would change in response to the increased perceived importance of a secondary task $\left(w\right)$ . At low values of $w$ , corresponding to conditions where the driver prioritizes driving safety and places little value on the secondary task, the predicted behavior is dominated by the safest action: maintaining on-road gaze or making only brief saccades. Thus, the green curve remains near one, while the probabilities of both short and long off-road glances remain negligible.

Figure 11.

Predicted effect of increasing driver’s perceived importance of secondary task (w).

As the weight increases toward the midrange $w=0.5$ , the driver begins to place comparable importance on benefit and cost. In this balanced trade-off region, the model predicts a rise in the probability of short off-road glances, indicated by the purple curve peaking. This reflects a strategic compromise in which the driver momentarily disengages from the road to attend to the secondary task but avoids prolonged distraction.

At high values of $w$ , where task benefits outweigh driving cost, the model predicts a shift toward prolonged off-road glances. The red curve increases steadily, indicating an increased likelihood of long glances away from the road and corresponds to a task-driven attentional policy.

Additionally, counterfactual simulations can be used to explore how drivers may adjust their attention in response to specific IVIS tasks with varying visual demands. As noted earlier, tasks like reading messages or entering navigation require sustained attention and high $B\left(a\right)$ , while brief tasks like toggling the radio have low $B\left(a\right)$ due to limited benefit. Figure 12 presents the results of a simulation illustrating how the predicted probability of prolonged off-road glances $\left(a=2\right)$ varies as a function of two key parameters: the perceived benefit of attention to the secondary task $\left(B\left(a\right)\right)$ and the weight assigned to this benefit ( $w$ ).

Figure 12.

The x-axis represents $B\left(a\right)$ , which captures the benefit of prolonged glances for successfully completion of secondary task. The y-axis reflects ( $w$ ), indicating the degree to which the driver prioritizes secondary task benefit over driving cost. The z-axis shows the predicted probability of the driver selecting prolonged off-road glance (action 2), visualized both as surface height and color gradient.

The surface increases monotonically along both the $B\left(a\right)$ and $w$ dimensions. When both parameters are low, the predicted probability of prolonged off-road glances remains near zero, indicating a strong preference for safety-oriented behavior. As either $B\left(a\right)$ or $w$ increases, the likelihood of choosing a prolonged glance also increases, with the highest probabilities observed in the top-right region of the plot. Notably, even when benefit $B\left(a\right)$ is moderate, drivers with a high $w$ exhibit elevated risk of distraction-prone behavior, suggesting that off-road glance behavior is highly sensitive to the internal valuation of benefit versus cost. This pattern is particularly relevant for young male drivers, who are frequently characterized in the literature as poorly calibrated, often underestimating the costs associated with inattention and engaging in riskier driving behaviors (Horrey et al., 2008).

Irrespective of the underlying cause, a driver profile characterized by high weight for secondary task benefit is concerning. Prolonged off-road glances are linked to double the risk of safety-critical events, increased lane deviation, slower braking responses, and reduced situational awareness (Dingus et al., 1989; Klauer et al., 2006; Liang et al., 2012; Victor et al., 2015). Existing research has established mechanisms such as graded warning systems, intervention strategies that escalate in intensity based on the inferred severity of distraction or safety risk (Lee et al., 2004). Our framework lends itself naturally to the development of graded warning systems. For individuals with low priority for secondary task engagement, the system could deliver subtle cues such as a visual prompt or a brief audio tone. Conversely, for drivers with a high $w$ profile, the system could implement strong interventions, such as freezing the secondary task application, or even advising the driver to pull over. Pre-knowledge of such individual reward profiles can enhance existing threshold-based graded warning systems (Forster et al., 2024; Li et al., 2022b; Shariatmadari et al., 2025) by enabling earlier and more context-based intervention, thus enhancing both responsiveness and personalization. Thus, our reward-based model has broad applications, from guiding targeted interventions to evaluating how a particular in-car interface design and driving context influence attention allocation.

Discussions

Contributions and Relation to Existing Computational Models of Distraction

The goal of this paper was to develop a computational account of how drivers allocate attention between the roadway and secondary IVIS tasks when their level of distraction is uncertain. This perspective differs fundamentally from prior computational models of driver distraction, which have largely focused on cognitive architectures and queuing network formulations of workload. The Distract-R system and the related general executive models (Salvucci, 2005, 2013) adopt a generative, production-rule approach in which a normative “average” agent interleaves tasks based on a goal queue. Although this approach formalizes how task constraints influence multitasking, it relies on simple, fixed rules that fail to explain how adaptive strategies emerge. Similarly, Queuing Network–Model Human Processor (QN-MHP) models distraction and warning responses as a problem of network congestion and information routing (Zhang et al., 2022; Wu & Liu, 2007). In QN-MHP, performance decrements arise when neural entities compete for processing servers, and task urgency is handled via priority classes (derived from lead time thresholds). This approach focuses on capacity limits rather than strategic choices. Consequently, it cannot predict how drivers dynamically adjust their task-interleaving behavior in response to evolving uncertainty.

Our POSMDP framework extends this literature in three key ways. First, we view the driver not merely as a processor subject to network congestion but as a rational agent who optimizes a utility function balancing safety and secondary-task benefit. Second, rather than simulating a normative agent, we estimate the specific reward weights ( $w$ ) of an individual driver, capturing heterogeneity in risk tolerance that rule-based architectures often abstract away. Third, attentional state is explicitly modeled as hidden and inferred probabilistically from multiple noisy behavioral and physiological cues, rather than assumed to be directly observable from gaze or task engagement alone. Although off-road glances are a direct and observable indicator of visual behavior, they do not fully capture the driver’s underlying attentional state. A brief glance away from the road does not necessarily imply loss of driving awareness, just as continuous gaze on the road does not guarantee cognitive engagement. The proposed model therefore integrates multiple noisy cues such as velocity, pupil-change dynamics, and glance behavior to infer a latent attentional state that reflects the driver’s internal engagement rather than gaze direction alone.

Our framework shares conceptual similarities with active inference approaches (Engström et al., 2024; Wei et al., 2025) in its emphasis on inference under uncertainty, but differs in both objective and application. However, the work of Wei et al., 2025 is closest to ours. That study fits an active inference model of decision making using observational data on AV takeovers together with self-reported measures of fatigue, trust, and situation awareness. A key distinction in our experimental design is that we directly manipulate distraction through a controlled secondary task. This manipulation provides an objective proxy for attentional load and thus a form of ground-truth supervision that is not available in purely observational settings, enabling sharper identification and validation of the inferred latent attentional state. In Engström et al., 2024, human driving maneuvers are modeled within an active inference framework as uncertainty-reducing actions, highlighting the role of epistemic objectives in shaping control behavior. The work does not address the task of estimating the parameters of such model. The estimation of an active inference model for car-following task is studied in Wei, Garcia, et al., 2025.

Finally, our model enables counterfactual simulations of how perceived task importance influences glance behavior. The findings have direct implications for the design of intelligent vehicle systems aimed at mitigating distraction. By inferring latent states and individual attention strategies, the framework supports adaptive, human-centered driver assistance systems that deliver graded, context-aware warnings and dynamically manage visually demanding IVIS interactions when distraction risk is high. Individualized utility profiles further allow tailoring interface complexity, feedback, and automation to driver-specific tendencies, improving both the timeliness and acceptance of interventions. Together, these contributions establish an interpretable and personalized modeling framework for understanding driver attention under multitasking.

Limitations and Future Work

We make simplifying assumptions to ensure model tractability, notably by assuming independence between observations (velocity and PCPS) by setting covariance terms to zero. A more sophisticated extension could exploit these interdependencies. Additionally, the current setup relies on velocity and PCPS as indicators of distraction and workload based on the available experimental design. While our model can be extended to incorporate other driving performance measures, those indicators are not significant in our current experimental setup. For example, reaction time is typically assessed using sudden-onset stimuli (e.g., braking lead vehicle) (Strayer et al., 2013). However, our experiment does not have such critical events. Future study could incorporate additional observations to provide a more comprehensive view of driver state. Future work could extend the POSMDP model to capture auditory and/or cognitive distraction by modifying the observation model. All participants in the present study were young male drivers. Although gender was not restricted during recruitment, only male volunteers enrolled and completed the study. It would be valuable and interesting for future research to explore how attention allocation strategies and multitasking behaviors may differ across diverse demographic groups, including various ages, genders, and cultural backgrounds, thus broadening the understanding of driver distraction. Environmental variation, such as differing traffic conditions or road types, could also be integrated to better understand attentional resource allocation in more realistic driving contexts.

An interesting avenue for further research is to infer a larger number of latent driver states, such as fatigue or drowsiness, offering deeper insights into driver behavior. Pursuing these extensions would require experimental protocols tailored to reliably evoke and distinguish among these additional states.

Conclusions

We present a computational framework for modeling and estimating driver attention and multitasking behavior under distraction. By integrating semi-Markov dynamics into a decision-theoretic framework, our model accounts for the temporal variability of attentional states, overcoming the limitations in standard POMDPs. Our framework not only infers the hidden state of the driver but also quantifies the perceived costs and benefits of secondary task completion that evolve with the state of the driver. The results highlight our ability to capture individual differences in multitasking strategies, and structured inference enables the design of context-sensitive, graded interventions that enhance driver safety.

Key Points

• We introduce a personalized driver distraction model using a Partially Observable Semi-Markov Decision Process (POSMDP).

• Model estimation recovers individual utility weights that capture how drivers trade off secondary task benefits against driving safety.

• POSMDP model outperforms traditional glance-duration thresholds in detecting distraction events and lane deviations.

• Our structural model enables counterfactual simulations to predict attention behavior under varying task demands and driver profiles.

Supplemental Material

Supplemental Material - Inferring Hidden Attentional States in Driving: A Bayesian Approach to Modeling Distraction and Secondary Task Engagement

Supplemental Material for Inferring Hidden Attentional States in Driving: A Bayesian Approach to Modeling Distraction and Secondary Task Engagement by Lekhapriya Dheeraj Kashyap, Zhide Wang, Yanling Chang, Maryam Zahabi, and Alfredo Garcia in Human Factors.