A Structural Model of Attentional Effort Dynamics: Evidence From a Naturalistic Discrimination Task

Abstract

Objective

To propose a model of how attentional effort varies over time in a vigilance task and how this effort relates to subjectively inferred context. To propose an estimation methodology and test the empirical validity of the proposed model in a naturalistic dataset.

Background

Attentional effort in a task can vary based on how an individual subjectively perceives the task context. However, both attention exertion and subjective context perception are not directly observable. We present a methodology for estimating a structural model that explicitly incorporates subjective models of context perception and attention allocation policies. To our knowledge, this is the first methodology to estimate a structural model of attentional effort dynamics.

Method

A Bayesian model of attentional allocation that integrates subjective perceptions of task-relevant context is developed. An estimation methodology based upon expectation-maximization algorithm is proposed to uncover how the allocation of attentional effort is adapted to subjectively perceived context.

Results

The methodology is applied to a naturalistic dataset of Major League Baseball umpire decisions, revealing context perception (i.e., how umpires infer game situations) and attention allocation policy (i.e., how umpires adjust attentional effort). Model reveals that umpires adjust attentional effort based on inferred game criticality and status bias.

Conclusion

This work advances understanding of vigilance failure by providing a structural account for contextual inference determines attentional effort. The estimated model closely tracks empirically observed decision accuracy patterns in a naturalistic dataset.

Application

The proposed model enables counterfactual predictions, allowing exploration of hypothetical interventions to improve decision accuracy in environments that require sustained attention.

Introduction

Tasks that require sustained attention over prolonged periods, are prone to performance degradation, and the resulting errors often stem from lapses in attention by human operators (Hancock, 2017; Warm et al., 2008). These errors range from minor oversights to serious failures, as seen in radar monitoring (Mackworth, 1950), quality control (Badalamente & Ayoub, 1969), airport baggage screening (Ghylin et al., 2007), and radiological image interpretation (Taylor-Phillips et al., 2015). While classic theories attribute declines in sustained attention to the depletion of finite cognitive resources over time (Grier et al., 2003; Warm et al., 2008), recent evidence challenges this view. For instance, in a study of expert radiologists performing prolonged breast cancer detection tasks, diagnostic accuracy was found to improve with time on task (Taylor-Phillips et al., 2024). Recent models argue instead that attentional effort is dynamically and strategically allocated based on perceived reward, motivation, and cost, rather than diminishing resource availability (Esterman & Rothlein, 2019; Fortenbaugh et al., 2017; Hockey, 2011; Wang et al., 2022).

A central factor in this strategic allocation of attentional effort is context. Rather than being shaped solely by cognitive load or task demands, research has shown that contextual learning can rapidly and automatically influence the instantiation of a given attentional set, indicating that attentional control can be modulated by learned contextual cues (Cosman & Vecera, 2013). Studies show that attention guidance is dependent on stimuli appearing in behaviorally relevant contexts (Britt et al., 2025). In fact, the notion of context is key to explaining how humans adaptively perform complex learning and memory tasks, making context-driven behavior a longstanding focus in psychology and neuroscience (Gershman, 2017). For example, context-dependent cognitive processing underlies human abilities to perform complex tasks, influencing spatial navigation (Gulli & others, 2020; Julian & Doeller, 2021; Plitt & Giocomo, 2021), problem solving strategies (Weitnauer et al., 2023), motor planning and execution (Heald et al., 2018, 2021; Wolpert & Kawato, 1998), learning and decision making (Geerts et al., 2024), cognitive control mechanisms (A. G. Collins & Frank, 2013), and memory structure and generalization (Franklin et al., 2020).

Although context profoundly influences cognitive processes and decisions, it is frequently hidden or only indirectly observable, reflecting subjective perceptions, beliefs, and internal states that are resistant to direct measurement (Heald et al., 2023). For example, in professional sports officiating, an umpire’s internal sense of game importance or momentary bias, although not directly observable, can influence vigilance fluctuations and consequently their judgment and accuracy. These internal states are often reflected in situational features such as pitch count or game score, which act as external cues to the underlying context. A growing body of research has formalized inference processes using probabilistic models, showing that people rely on environmental cues to infer such latent structure (Gershman, 2017; Gershman & Beck, 2017; Heald et al., 2023). Yet, the question of how these inferred beliefs influence the deployment of cognitive resources, such as attention or effort, remains less well understood. While studies show that reward effects on attention are context-dependent (Bourgeois et al., 2016; Pessoa, 2015), the context is usually predefined by the experimenter (e.g., a reward cue or task condition), limiting the relevance of the model to naturalistic environments. Further, few existing frameworks formally link the inferred context to control processes, limiting our ability to model and predict how individuals regulate attentional effort in response to situational demands.

This work seeks to bridge this theoretical gap by proposing a Bayesian framework that integrates context-inference models with mechanisms of attention regulation in vigilance tasks. Building upon Gershman’s probabilistic framework of context-dependent learning (Gershman, 2017) and theories of attentional effort grounded in executive control and strategic allocation (Kurzban et al., 2013; Thomson et al., 2015), we propose that individuals form subjective beliefs about the context based on observable task features, and dynamically adjust attentional effort according to these inferred contexts. Throughout this paper, we use the term “attention” to denote vigilance, understood as the regulation of cognitive engagement over time, rather than spatial attention or orienting.

Overview of the Proposed Framework and Analysis

Figure 1 provides a conceptual overview of our model. The agent has access to observations $z_t$ , which include task-relevant cues. The agent has an internal model of the world linking these observations to a context variable $s_t$ . Upon observing $z_{t-1}$ , the agent updates a belief distribution $x_t$ over the context $s_{t-1}$ and implements the attention policy $\pi$ , that is, the attention level is $a_t$ is sampled from a probabilistic policy $\pi \left(a_t|x_t\right)$ based on the updated belief $x_t$ . The context variable transitions to $s_t$ from $s_{t-1}$ according to a hidden Markov model and emits a new observation $z_t$ . The modeler (not the agent) has access to an objective assessment of task performance denoted by $u_t$ . In our proposed modeling framework, attention adapts to context, with observations informing contextual inference. Higher attention (quantified by an increase in relative entropy relative to a default minimal attention policy; see (Gershman, 2020; Lai & Gershman, 2024)) is associated with high task performance, other factors controlled. We develop a two-stage estimation methodology to uncover model parameters based on observables. First, we use the well-established Baum-Welch algorithm (Baum et al., 1970) to estimate the hidden context dynamics. In the second stage, we introduce a variation of the expectation-maximization (EM) algorithm (Dempster et al., 1977) to estimate the attention allocation policy as a function of relative reward trade-offs.

Figure 1.

Contextual inference and attention control.

Application to Naturalistic Discrimination Task Dataset

We apply our method to a naturalistic dataset of Major League Baseball (MLB) home plate umpire calls to investigate the dynamics of attentional effort. In baseball, home plate umpires must classify each pitch as either a “ball” (outside the strike zone) or a “strike” (within the strike zone); see Figure 2(a). Despite extensive training, umpires occasionally make incorrect calls. MLB publicly provides pitch-level data via the PITCHf/x system, which precisely tracks pitch trajectories and various game features. Crucially, the dataset includes ground-truth classifications derived from high-resolution camera systems (MLB, 2025).

Figure 2.

Results from our model of attentional allocation. (a) Visualization of a strike zone with umpire call errors from a real game. The black outline represents the official strike zone boundary. The orange area highlights the region close to the strike zone boundary where most umpire call errors occur. (b) The plot shows the probability of an umpire (Umpire II in this study) making a correct call as a function of the distance from the strike zone boundary (x-axis), comparing predictions from the high-attention model (green) and low-attention model (red).

A substantial body of research explores how observed accuracy correlates with observable factors such as pitch speed, location, and pitcher status (Archsmith et al., 2025; Flannagan et al., 2024; Kim & King, 2014; Mills, 2014; Parsons et al., 2011). For example, Archsmith et al. (2025) models observed accuracy as a function of “leverage,” a composite measure capturing the stakes of each play. The authors argue that attention operates as a depletable resource, citing evidence that high effort early in a game correlates with increased errors later on.

However, this interpretation is problematic for at least two reasons. First, observed accuracy is not a direct measure of attentional effort, which remains an inherently latent construct. Second, the observed correlation between leverage and accuracy may simply reflect changing game conditions rather than shifts in effort. In high-leverage situations, pitchers often target the strike zone’s edges more aggressively, making pitch calls more difficult. The resulting higher error rates may stem from task complexity, not reduced attentional engagement. Therefore, any model linking accuracy to strategic effort modulation must carefully account for such contextual confounds.

Our structural model addresses these issues directly by incorporating (i) an attention allocation policy that describes how umpires dynamically adjust effort based on inferred context, shown in Figure 2(b) and (ii) a model of subjective context perception that captures how umpires internally represent game situations, shown in Figure 3. Our findings show that high attentional engagement yields high observed accuracy regardless of task difficulty. In Figure 2(b), a high-attention policy (green line) leads to high accuracy even for borderline pitch locations, whereas a low-attention policy (red line) consistently results in lower accuracy. This pattern aligns with sustained attention theories linking mind wandering to performance declines (Mrazek et al., 2012; Smallwood et al., 2004) and increased susceptibility to bias (Demanet et al., 2013).

Figure 3.

The figure shows related probabilities across different game contexts $s$ (x-axis): The bar plots for the probability of choosing the high-attention policy; The red (blue) dotted line for the estimated (empirical) probability of a correct call of pitches near the strike zone boundary. Note: (1) The y-axis does not start at 0 to highlight variability in umpire performance. Because accuracies of professional umpires are concentrated in the upper range, restricting the axis improves visibility of error differences. (2) This figure reports results for Umpire II, presented as an example in the Introduction. Analyses across all six studied umpires is shown later (see Figure 5).

Figure 3 lustrates how subjectively perceived context, according to our model, determines the level of attentional effort. For instance, in Type D contexts (dark grey bars), umpires are predicted to allocate high attention with high probability. In contrast, in other contexts (e.g., Type E in blue or Type A in light grey), the model predicts low-attention levels. A detailed discussion of these context types follows in the section “Evidence from a Naturalistic Dataset” below. Notably, unlike prior regression-based approaches (MacMahon & Starkes, 2008), our model identifies previously overlooked features, such as pitcher reputation (Type E), as important drivers of attention allocation.

Perhaps most significantly, our structural model enables counterfactual predictions, simulations of hypothetical scenarios not directly observed in the dataset. This includes, for example, the effects of targeted incentives designed to improve performance. Laboratory studies suggest that enhancing performance-based rewards can counteract vigilance decrements and sustain attention (Esterman et al., 2016; Massar et al., 2016). Our model offers a principled framework for forecasting the impact of such interventions.

In the following sections, we begin by introducing our model, which links subjective context inference to a reward-based attention policy. We then present a variation of the EM algorithm to estimate attention allocation, followed by the discussion of results from the naturalistic dataset and a counterfactual analysis illustrating the benefits of context-sensitive attention modeling.

A Model of Context-Dependent Attention Allocation

Let the vector $z_t\in Z$ represent the observable features or incoming sensory information of the task at time $t$ . The context in which a task is performed is modeled by a state variable $s_t\in S$ which is hidden to the decision maker (and the modeler). The variables $s_t$ and $z_t$ are imperfectly correlated. We assume task features and hidden context state evolve according to a Markov process. That is, $\left(z_{t+1},s_{t+1}\right)$ , conditioned on $\left(z_t,s_t\right)$ , is independent of the information prior to time $t$ . In the umpire setting, this Markov property can be justified by the fact that an umpire’s decision primarily depends on the present game context state rather than the complete history of game observables. We further consider the following dependency structure of $\left(z,s\right)$ which is widely adopted in the literature of hidden Markov models (HMMs) (Rabiner, 1989). Such a structure captures how the agent’s perception of baseball game situation evolves, and how such perception is correlated with observable features.

Assumption 1

The context $s_t$ evolves independently of observable factor $z_t$ :

$\mathrm{Pr}\left(z_{t+1},s_{t+1}|z_t,s_t\right)=O\left(z_{t+1}|s_{t+1}\right)T\left(s_{t+1}|s_t\right),$

(1)

where $O\left(z_{t+1}|s_{t+1}\right)$ is the observation probability and $T\left(s_{t+1}|s_t\right)$ is the state transition probability.

Let $h_t^z=\left(z_0,z_1,\dots ,z_t\right)$ denote the observed task features up to time $t$ . Let $x_t\left(s\right)=\mathrm{Pr}\left(s_t=s\mid h_t^z,x_0\right)$ denote the belief distribution (i.e., the posterior probability) of the context $s_t=s$ at time $t$ , given the history of observed task features $h_t^z$ and the prior belief $x_0$ . Upon receiving a new observation $z_{t+1}$ , the agent updates its belief distribution as follows according to Bayes’ rule:

$\begin{array}{l}{x}_{t+1}\left(s\right)\hspace{0.17em}=\frac{{\displaystyle \sum }_{s_t\in S}O\left(z_{t+1}|s_{t+1}=s\right)T\left(s_{t+1}=s|s_t\right)x_t\left(s_t\right)}{{\displaystyle \sum }_{s_t\in S}{\displaystyle \sum }_{s_{t+1}\in S}O\left(z_{t+1}|s_{t+1}\right)T\left(s_{t+1}|s_t\right)x_t\left(s_t\right)},\end{array}$

(2)

and $h_{t+1}^z=\left(h_t^z,z_{t+1}\right)$ .

Let $u_t\in U$ denote an objective, observed numerical rating of the agent’s performance in executing the vigilance task at time $t$ . Here, we focus on a binary classification task where $u_t=1$ is the agent performing a correct classification and $u_t=0$ for an incorrect classification. We denote by $a_t\in A$ the agent’s attentional effort in executing the task at time $t$ and assume the agent’s task performance is contingent upon the amount of attention allocated to the task. This is modeled by the probability distribution $\mu \left(u_t|z_t,a_t\right)$ conditional upon the current observation $z_t$ and the attention level $a_t$ allocated to the task. As an example, if the agent pays less attention then the agent’s task performance deteriorates. We use the following logit model specification:

$\mu \left(u_t=1|z_t,a_t\right)=\frac{1}{1+\exp \left(-{\beta }_{a_t}^{\top }{z}_t\right)},$

(3)

where ${\beta }_{a_t}$ is the coefficient vector corresponding to action $a_t$ , matching the dimension of the observation vector $z_t$ .

While objective task performance $u_t$ is observed, the attention choice $a_t\in A$ is often private information to the decision maker and difficult for the modeler to measure and observe (Mancas & Ferrera, 2016). To model the allocation of attention, we use a context-adapted attention policy $\pi \left(·|s\right)\in {\mathrm{\Delta }}^{\left|A\right|}$ where ${\mathrm{\Delta }}^{\left|A\right|}$ is the set of probability distributions with support $A$ and $\pi \left(a|s\right)$ is the probability of choosing attention effort $a\in A$ given the context $s\in S$ . Specifically, we posit the following attention allocation model:

$\pi \left(·|s_t\right):=\arg \underset{\widehat{\pi }\in {\mathrm{\Delta }}^{\left|A\right|}}{\max }\left[{\displaystyle \sum }_{a\in A}r\left(s_t,a\right)\widehat{\pi }\left(a|s_t\right)-\kappa {\mathcal{D}}_{KL}\left(\widehat{\pi }\left(·|s_t\right)\Vert {\pi }^0\left(·|s_t\right)\right)\right],$

(4)

where $r\left(s_t,a_t\right)$ is the agent’s subjective reward of exerting attentional effort $a_t\in A$ in context $s_t\in S$ and the cost of exerting attentional effort is described by the Kullback–Leibler divergence with respect to a default (low) attention policy ${\pi }^0$ :

${\mathcal{D}}_{KL}\left(\widehat{\pi }\left(·|s_t\right)\Vert {\pi }^0\left(·|s_t\right)\right)={\displaystyle \sum }_{a\in A}\widehat{\pi }\left(a|s_t\right)\log \left(\frac{\widehat{\pi }\left(a|s_t\right)}{{\pi }^0\left(a|s_t\right)}\right),$

(5)

and $\kappa >0$ is a parameter that controls the magnitude of higher attentional effort (Gershman, 2020; Lai & Gershman, 2024). The closed-form solution to (4) is as follows:

$\pi \left(a_t|s_t\right)=\frac{{\pi }^0\left(a|s_t\right)\exp \frac{1}{\kappa }r\left(s_t,a_t\right)}{{\displaystyle \sum }_{\stackrel{\sim }{a}\in A}{\pi }^0\left(\stackrel{\sim }{a}|s_t\right)\exp \frac{1}{\kappa }r\left(s_t,\stackrel{\sim }{a}\right)}.$

(6)

Note that as $\kappa \to 0^{+}$ (i.e., when attentional effort is costless) the model assumes the agent chooses the reward maximizing level of attention with high probability. Conversely, as $\kappa \to +\infty$ , the agent reverts to the default low-attention policy ${\pi }^0$ .

As the agent has a subjective model of the context state variable $s_t$ , we further assume the attention allocation policy is adapted to agent’s subjective beliefs over context. Hence, the belief-based attention allocation policy is given by $\pi \left(a_t\left|x_t\right)={\displaystyle \sum }_{s\in S}\pi \left(a_t\right|s_t=s\right)x_t\left(s\right).$ One may interpret $\pi \left(a_t|s_t=s\right)$ as a library of pretrained policies indexed by context. Based on its belief, the agent retrieves relevant policies and weighs them according to the inferred context distribution, forming the belief-based policy $\mathrm{\pi }\left({\mathrm{a}}_{\mathrm{t}}|{\mathrm{x}}_{\mathrm{t}}\right)$ that accounts for contextual uncertainty. As discussed in (Heald et al., 2023), this approach of mixing policies differs from some recent models of context-dependent learning (A. Collins & Koechlin, 2012) that express only the policy associated with the single most probable context, potentially leading to sub-optimal decision selection by ignoring uncertainty about the context.

To summarize, our model describes how an agent (e.g., the umpire) allocates attention for undertaking a task (e.g., classification). At every period, the agent perceives a set of visible cues about the task, denoted by $z_t$ and forms a subjective belief about the task-relevant context $s_t$ . Given this belief, the agent decides how much attentional effort $a_t$ to invest in task execution. Paying more attention generally increases the chance of satisfactory task performance (e.g., correct judgment $u_t=1$ ), but it also incurs a cognitive cost. The model formalizes this trade-off through an attention allocation policy that balances the expected reward from improved accuracy against the mental cost of higher effort. The model combines a Bayesian model of the agent’s contextual inference with a classical (frequentist) model of task accuracy $\mu \left(u_t|z_t,a_t\right)$ , which links attention to task difficulty (e.g., proximity to the strike zone boundary), to generate the predicted call outcome. In general, the model links (a) changing task cues, (b) evolving beliefs about contextual inference, (c) strategic allocation of attentional effort, and (d) observed performance outcomes. Figure 1 illustrates this flow from perception to decision.

Estimation Methodology

The primitives of the model described in the previous section can be stacked into a single vector parameter $\theta =\left(O,T,\beta ,r,x_0\right)$ , where $\beta ={\left({\beta }_a\right)}_{a\in A}$ . We here introduce a method to identify the model primitives $\theta$ from data. To this end, each data sequence is composed of all observable information by period $T$ , including task features $h_T^z$ and objective task performance ratings $h_T^u=\left(u_0,u_1,...,u_T\right)$ . Since the context sequence ${\left(s_t\right)}_{t=0}^T$ and attention allocation actions ${\left(a_t\right)}_{t=0}^T$ are not observed, they are not included in the dataset. We search for the model parameters that maximize the log-likelihood of such data sequences

$\begin{array}{ll}\mathcal{l}\left(\theta \right)& =\log \hspace{0.17em}\mathrm{Pr}\left(h_T^u\cup h_T^z;\theta \right)=\log \hspace{0.17em}\mathrm{Pr}\left(h_T^u|h_T^z;\theta \right)+\log \hspace{0.17em}\mathrm{Pr}\left(h_T^z;{\theta }_z\right),\end{array}$

(7)

where ${\theta }_z=\left(O,T,x_0\right)$ represents parameters governing the dynamics of context states and observations. Since $\log \hspace{0.17em}\mathrm{Pr}\left(h_T^z;{\theta }_z\right)$ depends only on ${\theta }_z$ , we adopt a two-stage estimation approach. In the first stage, we estimate ${\theta }_z$ by maximizing $\log \hspace{0.17em}\mathrm{Pr}\left(h_T^z;{\theta }_z\right)$ . Once ${\theta }_z$ is identified, we use the Bayesian belief update rule (2) to construct a sequence of belief states ${\widehat{x}}_t=x_{t,{\widehat{\theta }}_z}$ from the observed history $h_T^z$ . This transformation allows us to express the decision process in terms of beliefs rather than raw observations, aligning with how the agent infers context in real time. Using these belief states, we estimate the remaining parameters ( $\beta ,r$ ) in the second stage by maximizing the likelihood function:

$\log \hspace{0.17em}\mathrm{Pr}\left(h_T^u|h_T^z;\theta \right)={\displaystyle \sum }_{t=0}^T\log \hspace{0.17em}\mathrm{Pr}\hspace{0.17em}\left(u_t|z_t,{\widehat{x}}_t;\beta ,r\right)={\displaystyle \sum }_{t=0}^T\log {\displaystyle \sum }_{a_t\in A}{\mu }_{\beta }\left(u_t|z_t,a_t\right){\pi }_r\left(a_t|{\widehat{x}}_t\right).$

(8)

Since $a_t$ is not directly observable, we apply EM method to approximately maximize the log-likelihood function $\mathcal{l}\left(\theta \right)$ by maximizing the evidence lower bound (ELBO). The derivation of the lower bound is relegated to Appendix A. It follows that:

$\begin{array}{ll}{\displaystyle \sum }_{t=0}^T\log \mathrm{Pr}\left(u_t|z_t,{\widehat{x}}_t;\beta ,r\right)& ={\displaystyle \sum }_{t=0}^T{\displaystyle \sum }_{a\in A}{q}_t\left(a\right)\hspace{0.17em}\log \frac{\mathrm{Pr}\left(a,u_t|z_t,{\widehat{x}}_t\right)}{q_t\left(a\right)}+{\displaystyle \sum }_{t=0}^T{D}_{KL}\left(q(·)\parallel \mathrm{Pr}\left(·|u_t,z_t,{\widehat{x}}_t\right)\right)\\ & \ge {\displaystyle \sum }_{t=0}^T{\displaystyle \sum }_{a\in A}{q}_t\left(a\right)\hspace{0.17em}\log \frac{{\mu }_{\beta }\left(u_t|a,z_t\right){\pi }_r\left(a|{\widehat{x}}_t\right)}{q_t\left(a\right)},\end{array}$

(9)

where the second inequality holds as the KL divergence is nonnegative; also note that the equality holds in the second line of the above inequality when the variational approximation $q$ equals $\mathrm{Pr}(·)$ . The EM algorithm (presented in Algorithm 1 in Appendix A) aims to iteratively maximize the lower bound in (9).

In our framework, Bayesian inference operates in two ways. At the behavioral level, it represents how the agent updates beliefs about latent contexts. At the methodological level, it is implemented through an EM-based procedure that approximates Bayesian inference over latent states and parameters. This Bayesian component is paired with a classical call precision model, in which the relationship between attention effort and decision precision ( $\mu$ ) is specified using a standard logit model rather than a Bayesian logit model.

Illustration with Synthetic Data

We test our model and estimation using a synthetic experiment with two contexts and a single observable variable $z$ in a sustained attention task; see Figure 4. The reward for choosing action $a$ depends on the unobserved context $s$ , as illustrated in Figure 4(a). The decision variable $a_t\in \left\{0,1\right\}$ represents, respectively, low or high level of attention. Under high-stakes context 1, allocating high attention ( $a_t=1)$ yields greater reward (1.5) compared to low attention (0), leading to an increased likelihood of a successful outcome $u_t=1$ ; see Figure 4(b). Whereas, in low-stakes context 2, high attention ( $a_t=1$ ) results in a lower reward (−2) than low attention (0), making exerting low attention ( $a_t=0$ ) the preferred choice. This setting reflects the situation where the performance gain from high attention is insufficient to justify its effort cost and low attention suffices for adequate performance given the context (Brocas & Carrillo, 2018). In line with resource control paradigm, the unrewarding nature of the vigilance task in context 2 diminishes motivational engagement, thereby increasing susceptibility to mind wandering and resulting in impaired task performance (Thomson et al., 2015); see Figure 4(b). The dynamics of the hidden contexts are summarized in Table 1.

Figure 4.

Illustrative Example of Attention and Task Policies with two hidden contexts. (a) Attention Policy (Reward function): In context 1, action $a_1$ yields higher reward than in context 2 for the same observation. (b) Task Policy: Performance (u) for low (a = 0) and high (a = 1) attention levels. High attention (a = 1) shows clearer distinction between correct (u = 1) and incorrect (u = 0) calls.

Table 1.

(a) Ground Truth and (b) Estimation Results of the Dynamics $T,O,x_0$ (Standard Error in Parenthesis).

(a) Ground truth model primitives
True Model
$x_0=\left[1.0,0.0\right]$
$T$			$O$
	$s_1$	$s_2$		${\mu }_z$	${\sigma }_z$
$s_1$	0.8	0.2	$s_1$	1	0.03
$s_2$	0.1	0.9	$s_2$	2.5	0.1

(b) Estimates of model primitives
Estimated Model
$\widehat{x_0}=\left[0.99,0.0\right]$
$\widehat{T}$			$\widehat{O}$
	$s_1$	$s_2$		${\mu }_z$	${\sigma }_z$
$s_1$	0.801 (0.0015)	0.199 (0.0015)	$s_1$	0.999 (<0.001)	0.031 (0.001)
$s_2$	0.1005 (<0.001)	0.8995 (<0.001)	$s_2$	2.499 (<0.001)	0.099 (<0.001)

We generate synthetic data using the ground truth dynamics (Table 1) and context-dependent rewards (Table 2), and then apply our estimation method to the synthetic data. For identifiability, the reward of action $a_0$ is fixed at $r\left(s,z,a_0\right)=0$ for all $s,z$ , allowing other utilities to be measured relative to this baseline (Chang et al., 2023; Magnac & Thesmar, 2002; Wang et al., 2022). Estimation results in Table 1 and Table 2 confirm accurate recovery of model primitives $\theta$ .

Table 2.

Ground Truth (a) and Estimation Results (b) for the Attention Reward $r\left(s,a\right)$ and Task Policy Parameter $\beta$ .

(a) Ground truth model primitives
True Model
$r$			$\beta$
	$s_1$	$s_2$		$a_0$	$a_1$
$r\left(s,a_0\right)$	0*	0*	${\beta }_a$	0.5	3.0
$r\left(s,a_1\right)$	1.5	−2

(b) Estimates of model primitives
Estimated Model
$\widehat{\pi }$			$\widehat{\beta }$
	$s_1$	$s_2$		$a_0$	$a_1$
$\widehat{r}\left(s,a_0\right)$	0*	0*	${\widehat{\beta }}_a$	0.498 (0.004)	3.05 (0.05)
$\widehat{r}\left(s,a_1\right)$	1.503 (0.02)	−2.06 (0.05)

Standard errors are in parenthesis and values with asterisk are the fixed reference action rewards.

Evidence from a Naturalistic Dataset: Umpire’s Classification Task in Major League Baseball

We obtain pitch-level data for all MLB games from seasons 2008 to 2022 from MLB’s website (MLB, 2025). This dataset allows us to study attentional processes in a natural, real-world setting, where decisions carry real consequences, providing insights that extend beyond what can typically be observed in controlled laboratory experiments. It includes game details such as the identities of the home/away team, the players and season type. The pitch level details include pitch type (fastball, curveball, etc.), situational variables (inning, runs, etc.) and pitch outcomes. We restrict our analysis to regular season games and eliminate games that are suspended (e.g., due to inclement weather) and games that go into extra innings. This decision was primarily made to ensure consistency in game duration and remove potential outliers.

Rather than aggregating data across umpires, we focus on individual umpire decisions. As of 2022, umpire accuracy ranges from $90\%-95\%$ . An umpire was selected, designated as Umpire II, whose 91.4% accuracy aligns with the 92% league average. Additionally, Umpire II has officiated 395 games, making over 57,500 pitch calls, ensuring a robust sample for analysis.

The Umpire Model

Observations for Contextual inference

Following recent studies on umpire accuracy (Green & Daniels, 2014; Kim & King, 2014; Mills, 2014), we consider game related factors ( $z^{\text{context}}$ ): the batter’s strike count ( $z_s$ ) and ball count ( $z_b$ ). We include a home team indicator ( $z_h$ ) to account for potential home team advantage. Specifically, $z_h=0$ indicates the top half and $z_h=1$ indicates the bottom half of an inning. We account for the status of the pitcher ( $z_p$ ) by a binary variable where 1 corresponds to a high-status of the pitcher (All-star player). An All-Star selection is a prestigious accolade that underscores a pitcher’s standing as one of the elite players in the league (Kim & King, 2014). Collectively, at each time step, the set of observations consists of $z_t^{\text{context}}=\left(z_b,z_s,z_p,z_h\right)$ . These observations were chosen a priori based on literature linking them to umpire decision making (Flannagan et al., 2024; Hsu, 2024; Kim & King, 2014; MacMahon & Starkes, 2008; Mills, 2014). We also examined other plausible factors (e.g., base-runner configuration), but found no meaningful association with call accuracy in our dataset, so they were excluded.

Attention Allocation

We consider umpire attention allocation as a binary action where $a=0$ represents low attention and $a=1$ denotes high attention. These attention action variables are unobserved and need be inferred.

Call Accuracy Model

To model call accuracy, we incorporate pitch characteristics that influence the relative difficulty of making accurate calls, such as pitch type, speed, and proximity to the strike zone boundary (Flannagan et al., 2024; Kim & King, 2014). Pitch types are represented by categorical variables $z_1,z_2$ and $z_3$ , that categorize pitches as breaking (curveball, slider, knuckle-curve), off-speed (changeup, knuckleball, split-finger), and fastballs (Four-seam, Fastball, Sinker), respectively. In addition, we quantify the pitch’s closeness to the strike zone as a continuous variable $z_4$ , measuring the absolute distance in units from the zone’s boundaries as the ball crosses the plate. Taken together $z^{\text{task}}=\left(z_1,z_2,z_3,z_4\right)$ . Therefore, the model includes five coefficients ( ${\beta }_{a,0}$ to ${\beta }_{a,4}$ ) for each attention state, where ${\beta }_{a,0}$ is the constant term, ${\beta }_{a,1}$ , ${\beta }_{a,2}$ , and ${\beta }_{a,3}$ represent dummy variables for breaking, off-speed, and fastball pitch types, respectively, and ${\beta }_4$ represents pitch distance from the strikezone boundary. Hence, the call accuracy is modeled by

$\mu \left(u_t=1|z_t^{\text{task}},a_t\right)=\frac{1}{1+\exp \left(-\left({\beta }_{a_t,0}+{\beta }_{a_t,1}{z}_1+{\beta }_{a_t,2}{z}_2+{\beta }_{a_t,3}{z}_3+{\beta }_{a_t,4}{z}_4\right)\right)}.$

(10)

Reward Function

We model the reward for each context action pair such that $r\left(s,a\right)={\theta }_{s,a}^r$ . This reward function explicitly models the context-dependent subjective value of effort in an attention control system.

Validation

We begin by presenting the model’s validation results, establishing its descriptive adequacy relative to observed umpire decisions. We compare the model’s estimated accuracy patterns (red dotted lines in Figure 3) with empirically observed call precision across different game situations (blue dotted lines). The close alignment between these estimates and real-world decisions suggests that our model effectively captures how attention allocation shapes call accuracy under varying contexts. To assess the model’s robustness, we apply it to six umpires (Umpires I–VI), ordered by career accuracy from 90% to 95%. Each accuracy level is represented by an umpire with a sufficiently large call volume to ensure reliable estimation. In the context inference phase, we find that the inferred latent contexts are highly consistent across umpires. The differences in the estimated dynamics are minimal, on the order of 1e-3, indicating that the model captures a shared underlying structure of game contexts.

Furthermore, the model’s estimated accuracy patterns consistently align with observed performance across individual umpires (see Figure 5). In addition, our estimated probability of correct calls increases across umpires in a way that aligns with their overall career accuracy. For instance, in context 1, the estimated accuracy values for pitches located at the boundary ( $z_4=0$ ) are 0.61, 0.62, 0.67, 0.69, 0.74, and 0.69 for Umpires I through VI, respectively. While each umpire displays distinct tendencies, Figure 6 highlights this broader trend, illustrating that Umpire I consistently perform below umpires with higher overall accuracy across all contexts.

Figure 5.

The bar plots represent the probability of choosing the high-attention policy, $\pi \left(a=1|s\right),$ across different game contexts. The red dotted line denotes the estimated probability of a correct call, $\mu \left(u=1|z,s\right)$ , for pitches near the strike zone boundary. The blue dotted line represents the empirical accuracy of the umpire for pitches close to the boundary. Error bars indicate standard errors. Note: (1) The y-axis does not start at 0 in order to highlight variability in umpire performance. Because accuracies of professional umpires are concentrated in the upper range, restricting the axis improves visibility of error differences. (2) Umpire II is not repeated here because their results were presented in detail in the Introduction (Figure 3).

Figure 6.

Estimated probability of correct calls (y-axis) by Umpires for pitches at the boundary across different game contexts (x-axis). The overall career decision accuracy is presented in parenthesis.

Estimation Results

For illustration purposes, we focus on the estimation results of Umpire II. The results for other umpires share a similar pattern, which can be found in Section Validation. The number of contexts is unknown a priori, so we estimated models with different numbers of hidden contexts, ranging from 2 to 20, and evaluated them using AIC, BIC, and likelihood scores. While increasing the number of contexts improves model fit, we aimed to balance model complexity and interpretability without overfitting the data. After assessing redundancy, context specificity, and estimation stability, we selected a model with 12 contexts as an interpretable and robust choice. The detailed model selection process is provided in the Appendix.

Context Profiling

Figure 7 shows how the 12 identified contexts are categorized into five distinct types (A–E) to enhance interpretability. As research suggests that umpires may exhibit different judgment when officiating pitches by high-status pitchers (Kim & King, 2014), Types A–D encompass game situations involving non-high-status pitchers (i.e., not All-Star players), whereas Type E represents all pitches from high-status players.

Figure 7.

Classification of Game Contexts identified for Umpire II dataset. This figure illustrates the categorization of 12 identified game contexts into 5 distinct types (A–E). Types A–D, represented by varying shades of grey, depict increasing levels of game criticality for non-high-status pitchers. Type E, shown in blue, represents situations involving high-status pitchers. Each context type is further divided based on the side of the inning (top or bottom). The top three most probable observations for each context are displayed in the table, ordered by their probability of occurrence.

In general, Types A–D reflect increasing criticality of the game situation. Specifically, Type A covers the start of a batter’s turn; Type B includes situations with either 1 strike or 1 ball count (but not both); Type C primarily comprises even count (1-1), additionally includes counts 2-0 and 0-2; and Type D includes critical and decisive counts such as 3-2, 1-2, 2-2, 2-1, and 3-1. These counts are critical moments in the at-bat where the next pitch can significantly alter the game’s momentum, often resulting in immediate and decisive outcomes, which requires both the pitcher and the batter to make strategic decisions under pressure. For example, pitchers tend to deliver their fastest balls during the most critical counts (Lee et al., 2024). Each context type is further subdivided based on the inning side, that is, pitches in the top of the inning or bottom of the inning. Table 3 showcases the observation probability under each context type.

Table 3.

Contexts, Observations and Observation Probabilities.

Context Type	Context	Ball #	Strike #	Pitcher Status	Home Batter	$O=\mathrm{Pr}\left(z_t|s_t\right)$
A	$s_1$	0	0	0	0	1.00
A	$s_2$	0	0	0	1	1.00
B	$s_3$	0	1	0	0	0.55
B	$s_3$	1	0	0	0	0.45
B	$s_4$	0	1	0	1	0.55
B	$s_4$	1	0	0	1	0.45
C	$s_5$	1	1	0	0	0.54
C	$s_5$	0	2	0	0	0.27
C	$s_5$	2	0	0	0	0.19
C	$s_6$	1	1	0	1	0.55
C	$s_6$	0	2	0	1	0.25
C	$s_6$	2	0	0	1	0.19
D	$s_7$	1	2	0	0	0.30
D	$s_7$	2	2	0	0	0.25
D	$s_7$	2	1	0	0	0.16
D	$s_7$	3	2	0	0	0.15
D	$s_8$	1	2	0	1	0.54
D	$s_8$	2	1	0	1	0.32
D	$s_9$	2	2	0	1	0.53
D	$s_9$	3	2	0	1	0.32
D	$s_9$	3	1	0	1	0.15
E	$s_{10}$	0	0	1	0	0.45
E	$s_{10}$	1	1	1	0	0.18
E	$s_{10}$	2	2	1	0	0.12
E	$s_{11}$	0	1	1	0	0.31
E	$s_{11}$	1	0	1	0	0.24
E	$s_{11}$	1	2	1	0	0.19
E	$s_{11}$	2	1	1	0	0.12
E	$s_{12}$	0	0	1	1	0.26
E	$s_{12}$	0	1	1	1	0.13

For clarity only probabilities ≥ 10% are shown. Note that Context 12 has a more uniform distribution across many observations, only the two highest probabilities are shown here.

Figure 8 illustrates the transitions of unobserved context within an inning across different pitcher status scenarios. Figure 8(a) and 8(c) represent top and bottom of the inning, respectively, played by non-high-status pitchers. In these cases, context progresses from Type A to Type D, reflecting increasing criticality as the at-bat advances. Figure 8(b) and 8(d) represent top and bottom of the inning, respectively, played by high-status pitchers. Notably, in these scenarios, the context remains relatively stable, with the type not changing for extended periods. This stability justifies dedicating a separate type (Type E) to high-status pitchers.

Figure 8.

Transitions of Umpire Decision Contexts During a Game. This figure displays the transitions of unobserved contexts throughout an inning, based on pitches from a real game officiated by umpire II. The y-axis represents the 12 identified contexts, while the x-axis shows the sequence of pitches. The progression from Type A to Type D reflects increasing criticality in at-bats, while high-status pitchers exhibit more stable context dynamics compared to their non-high-status counterparts.

Attention Allocation Policy

The estimates of rewards that describe attention policy for different contexts are summarized in Table 4. We can see that the motivation for exerting attention effort increases with the increase of the game situation criticality. Type D, the most critical situation among types, has the highest estimated reward values. In contrast, Type A shows relatively low estimated rewards, suggesting that Type A is less impactful therefore demands less attention. Type D differs from Types A–C as its bottom of the inning is represented by two context states (8 and 9) instead of one, suggesting that umpires may perceive high-stakes situations more granularly, potentially indicating heightened scrutiny and adaptive decision making. Type E, on the other hand, requires separate analysis as it exclusively involves contexts with high-status pitchers. The presence of a high-status pitcher likely alters the umpire’s perception, suggesting that the impact of pitcher status on umpire behavior may override the general tendency observed in other context types. This finding aligns with prior research which found that umpires tend to expand the strike zone for high-status pitchers (Kim & King, 2014).

Table 4.

Reward Estimates ${\theta }_{s,a=1}^r$ and Standard Errors in Parentheses Across Different Contexts for Umpire II $({\theta }_{s,a=0}^r$ Is Normalized to 0).

Context	Type A		Type B		Type C		Type D			Type E
	1	2	3	4	5	6	7	8	9	10	11	12
${\theta }_{s,a=1}^r$	0.33 (0.02)	0.48 (0.03)	0.52 (0.03)	0.55 (0.04)	0.76 (0.04)	0.71 (0.04)	0.93 (0.04)	1.18 (0.06)	1.07 (0.05)	0.54 (0.03)	0.47 (0.02)	0.66 (0.03)

Task Performance

Figure 2b in the Introduction Section and Table 5 show the results from the call accuracy model, estimated using the EM algorithm, which reveal distinct patterns in umpire call accuracy under low and high attention states. In Table 5 we report Bayes factors ( $\text{BF}$ ) (Rouder et al., 2009), which measure the evidence for a nonzero coefficient ( $H_1$ ) relative to the null hypothesis of no effect ( $H_0$ ). As shown in Figure 2(b), in the low-attention model, pitch location significantly impacts call accuracy, with pitches closer to the strike zone boundary more likely to result in errors (see Table 5 with $p<0.001$ and $\text{BF}>150$ , decisively favoring $H_1$ ). This relationship shows a sharp decline in error probability as the distance from the boundary increases. Fastballs and off-speed pitch types negatively affect correct call likelihood ( $p<0.05$ ), with corresponding $\text{BF}$ values of approximately 2, indicating only anecdotal evidence for $H_1$ . The other pitch types show no statistically significant effects and their Bayes factors are near 1, providing evidence in favor of $H_0$ . In contrast, the high-attention model demonstrates a reduced impact of pitch location on errors, though still significant ( $p<0.001$ ), with $\text{BF}>150$ offering decisive evidence for $H_1$ . This suggests that when umpires allocate high attentional effort, they are more likely to make correct calls even for challenging pitches. More importantly, in the high-attention state, we no longer find a statistically significant effect for pitch types ( $p>0.05$ and $\text{BF}$ below or near 1), this failure to reject the null hypothesis is consistent with the idea that heightened focus may mitigate the challenges posed by different pitch characteristics. Figure 2(b) illustrates how task difficulty, reflected by proximity to the strike zone boundary, influences decision accuracy under varying levels of attentional engagement. Consistent with vigilance theory, low attention or mind wandering is associated with elevated error rates.

Table 5.

The Coefficients ${\beta }_0,{\beta }_1,{\beta }_2,{\beta }_3,{\beta }_4$ Represent a Constant, the Breaking, Off-Speed, and Fastball Pitch Types, the Distance From the Boundary of the Strike Zone, Respectively.

	${\beta }_{a,0}$	${\beta }_{a,1}$	${\beta }_{a,2}$	${\beta }_{a,3}$	${\beta }_{a,4}$
Low attention	−0.09	−0.95	−1.09*	−1.02*	8.70**
$a=0$	(0.34)	(1.15)	(2.27)	(2.33)	(>150)
High attention	1.24**	1.65	4.03	0.47	5.44**
$a=1$	(4.04)	(0.32)	(1.06)	(0.37)	(>150)

Bayes factor (BF) are in parentheses. **p < 0.01, *p < 0.05.

By combining the estimation results for both dynamics and rewards, we find that both contextual factors $z^{\text{context}}$ and attention related observations $z^{\text{task}}$ can both influence an umpire’s accuracy. For instance, for Type D, for which an umpire tends to exert attention (e.g., 76.5% for game context 8), the probability of an accurate call is 76.7% for a near boundary pitch; this probability increases to 99% when the pitch is far from the boundary. Conversely, Type A, associated with a low probability of attention (e.g., 58.2% for game context 1), has a 63.1% accuracy rate for near boundary pitches, which improves to 99% for pitches farther from the boundary. As such, attention exertion impacts the accuracy the most for close boundary pitches, and such impacts diminishes when pitches move further away from boundary. Interestingly, the call accuracy was found to have dropped for noncritical situations compared to critical ones, suggesting a potential insufficient attentional effort of an umpire in noncritical situations.

In a nutshell, these estimation results reveal that umpires adjust attentional effort in systematic and strategically interpretable ways: increasing effort as game situations become more consequential, and responding differently to contextual cues such as ball count and pitcher reputation. These behavioral patterns align with our model’s core premise that attention allocation in a vigilance task is sensitive to inferred task importance and cognitive biases. Importantly, the latent structures inferred by the model were stable across individuals, while still allowing for individual variation in policy and performance, suggesting a shared underlying attentional framework modulated by agent-specific strategies. This supports the value of structural modeling in revealing how context inference and sustained effort allocation jointly shape real-world decision accuracy, beyond what traditional statistical models can capture.

Discussion

Incentive-Based Interventions on Decision Accuracy

The 2009 collective bargaining agreement between MLB and the umpire union introduced performance incentives based on PITCHf/x data to motivate umpires to call strikes more consistently with the official rulebook definition. That said, these incentives are restrictive in that they cannot adapt to other factors in addition to umpire performance, such as player status induced performance biases. Our model can inform quantitatively how to reduce player type related biases in umpire officiating games, by offering monetary incentives for certain type of players. Consider umpire II, whose call accuracy against high-status pitchers (Type E) is 87% (which is relatively low), showing that umpires tend to favor elite pitchers by deviating from the optimal strike zone (Kim & King, 2014). To counter this, we introduce a hypothetical incentive that increases the reward of Type E decisions to 4 (from 0.54, 0.47, and 0.66 for contexts 10–12), raising the perceived benefit of exerting attention effort and making correct calls in this context. Under these adjusted rewards, the model predicts a revised attention and task execution policy, which results in a substantial improvement in accuracy from 87.05% to 98.19%, as shown in Figure 9. This shows that targeted incentives can effectively attenuate biases, leading to a more equitable and rulebook-consistent officiating process, and is aligned with studies showing that umpires’ ball-strike calling patterns have shifted in alignment with the implementation of these monitoring and training enhancements (Flannagan et al., 2024; Mills, 2017). Lab-controlled studies manipulating motivation have provided support for this, showing that performance-based rewards can enhance sustained attention (Esterman et al., 2016; Massar et al., 2016). In particular, maintaining consistent motivation, such as through the anticipation of a potential reward, has been shown to eliminate vigilance decrements and significantly reduce attentional lapses.

Figure 9.

Predicted umpire decision accuracy under varying levels of incentive-based rewards for calls involving high-status (Type E) pitchers: The x-axis represents the hypothetical reward values assigned to calls in these contexts, while the y-axis shows the model-predicted accuracy. As the perceived benefit of exerting attentional effort increases, so does the predicted accuracy, illustrating the potential effectiveness of targeted incentives. Note: The y-axis does not start at 0 to highlight variability in umpire performance. Because accuracies of professional umpires are concentrated in the upper range, restricting the axis improves visibility of error differences.

Cross-Agent Heterogeneity in Attention and Decision Policies

In the Results Section, we validated the stability of contextual inference and task models across umpires, we now examine how individual umpires differ in their inferred attention and task policies. Most umpires exhibit higher estimated reward for Type D contexts (contexts 7–9), consistent with greater attention in high-stakes situations (Table 6). However, notable deviations appear. Umpire shows particularly high preference for exerting attention for context 4 (Type B) and 5 (Type C), and Umpire places much higher value of attentional effort on context 5 (Type C). These idiosyncrasies may reflect differences in experience, attention strategy, or personal heuristics, pointing to heterogeneity in how umpires perceive and respond to game situations.

Table 6.

Reward Estimates ${\theta }_{s,a=1}^r$ Across Different Contexts for 6 Umpires Ordered by Their Empirically Observed Call Accuracy.

Umpire	Accuracy	Pitches Called	Context
			Type A		Type B		Type C		Type D			Type E
			1	2	3	4	5	6	7	8	9	10	11	12
I	90.90%	53.6k	0.59	0.72	0.77	0.87	0.94	1.01	1.38	1.36	1.27	0.59	0.76	0.75
			(0.07)	(0.07)	(0.06)	(0.07)	(0.08)	(0.08)	(0.08)	(0.08)	(0.06)	(0.07)	(0.07)	(0.07)
II	91.40%	57.7k	0.33	0.48	0.52	0.55	0.76	0.71	0.93	1.18	1.07	0.54	0.47	0.66
			(0.02)	(0.03)	(0.03)	(0.04)	(0.04)	(0.04)	(0.04)	(0.06)	(0.05)	(0.03)	(0.02)	(0.03)
III	91.80%	57.3k	0.75	0.66	0.79	0.87	1.00	0.87	1.32	1.12	1.30	0.70	0.86	0.73
			(0.01)	(0.02)	(0.03)	(0.01)	(0.02)	(0.03)	(0.03)	(0.04)	(0.05)	(0.03)	(0.01)	(0.02)
IV	93.00%	30.7k	0.53	0.46	0.68	0.65	0.84	0.85	0.90	1.05	0.49	0.59	0.76	0.80
			(0.05)	(0.03)	(0.04)	(0.04)	(0.07)	(0.05)	(0.06)	(0.05)	(0.06)	(0.04)	(0.07)	(0.05)
V	94.10%	30.6k	0.72	0.74	0.87	0.81	0.84	0.86	1.14	1.15	1.06	0.76	0.79	0.78
			(0.07)	(0.08)	(0.07)	(0.09)	(0.08)	(0.07)	(0.07)	(0.11)	(0.11)	(0.07)	(0.08)	(0.07)
VI	95.00%	13.8k	1.10	1.05	0.87	1.62	1.57	1.48	1.27	1.01	1.61	1.29	1.06	1.14
			(0.15)	(0.21)	(0.13)	(0.22)	(0.22)	(0.21)	(0.18)	(0.15)	(0.21)	(0.22)	(0.14)	(0.16)

For each umpire, three highest reward values are highlighted, showcasing the high importance Type D is associated with.

The task policy parameter $\beta$ estimates for six umpires (I through VI) are presented in Table 7. A consistent finding across all umpires and attention states is the strong positive and statistically significant ( $p<0.01$ , with corresponding Bayes factors supporting $H_1$ ) effect of pitch distance ( ${\beta }_4$ ). This suggests that as the distance from the strike zone boundary increases, the likelihood of a correct call rapidly improves. In the high-attention model, the constant term ( ${\beta }_0$ ) is statistically significant and positive, indicating that high attention leads more likely to correct calls. The pitch type variables ( ${\beta }_1,{\beta }_2,{\beta }_3$ ) generally less statistically significant, and in most cases their Bayes factors are near or below 1, supporting $H_0$ and suggesting that when the umpire is fully attended, the proximity of the pitches to the strike zone becomes the primary factor influencing call accuracy. The type of pitch (e.g., fastball), while still important in the physical dynamics of the game, does not significantly contribute to the correctness of a call. This result aligns with the notion that in high-stakes or high-attention situations, an umpire’s decision making is more influenced by the challenging nature of the pitch’s location than by the specific characteristics of the pitch type (e.g., be it a curveball or fastball).

Table 7.

Task Policy Estimates for 6 Umpires Ordered by Their Empirically Observed Call Accuracy.

Umpire	Accuracy	Pitches Called	Task Policy Estimates
			Low Attention $a=0$					High Attention $a=1$
			${\beta }_0$	${\beta }_1$	${\beta }_2$	${\beta }_3$	${\beta }_4$	${\beta }_0$	${\beta }_1$	${\beta }_2$	${\beta }_3$	${\beta }_4$
I	90.90%	53.6k	−1.7**	0.9	0.84	0.41	8.48**	2.41**	−0.58	−0.11	−0.75	8.31**
			(15.5)	(1.87)	(2.03)	(0.35)	(>150)	(5.16)	(0.57)	(0.37)	(0.72)	(>150)
II	91.40%	57.7k	−0.09	−0.95	−1.09*	−1.02*	8.70**	1.24*	1.65	4.03	0.47	5.44**
			(0.34)	(1.15)	(2.27)	(2.33)	(>150)	(4.04)	(0.32)	(1.06)	(0.37)	(>150)
III	91.80%	57.3k	0.32	−1.77*	−1.96*	−2.12*	10.18**	1.21*	1.13	17.19*	1.05	6.94**
			(0.37)	(2.23)	(2.55)	(3.1)	(>150)	(2.49)	(0.35)	(4.63)	(0.31)	(>150)
IV	93.00%	30.7k	2.82*	−3.31	−3.64	−4.0**	11.81**	0.36**	4.45	14.87	2.7	8.59**
			(2.58)	(0.53)	(0.36)	(13.18)	(>150)	(10.1)	(0.66)	(1.13)	(0.32)	(>150)
V	94.10%	30.6k	−0.60*	−0.55*	−0.33*	−0.91*	12.90**	10.80**	−2.62	3.45	−6.52	7.72**
			(2.22)	(3.58)	(4.33)	(5.56)	(>150)	(11.94)	(0.30)	(1.25)	(1.26)	(>150)
VI	95.00%	13.8k	−0.09*	−1.20*	−1.48	−0.85	12.26**	11.11*	−4.57	4.03	−5.88	12.24**
			(3.89)	(3.15)	(0.89)	(1.15)	(>150)	(16.46)	(1.06)	(2.02)	(1.17)	(>150)

Bayes factors (BF) are in parentheses. **p < 0.01, *p < 0.05.

Conversely, in the low-attention model, the effects of pitch types are more varied. When statistically significant, these variables often show a negative relationship with the likelihood of a correct call, with Umpire being an exception to this trend. This pattern aligns with attention control theories positing that attention functions as a regulatory mechanism, allocating resources more effectively in response to increased task complexity and uncertainty.

Contribution and Generalization Across Domains

Our model extends vigilance research by formalizing context as a central determinant of attentional effort. Context captures the task environment and situational conditions under which vigilance is sustained, offering a structural account of how both contextual and noncognitive factors (e.g., task difficulty) shape behavior and vary across individuals. Unlike existing approaches (Esterman & Rothlein, 2019; Fortenbaugh et al., 2017; Hockey, 2011), our framework estimates context and reward from observed behavior, treating attentional effort as a latent policy shaped by inferred utilities. This allows us to explicitly model how decision makers trade-off the costs and benefits of attentional engagement under varying circumstances. We model attention as a binary variable (low versus high), reflecting the well-established contrast between task engagement and lapses such as mind wandering (Mrazek et al., 2012; Smallwood et al., 2004). While interpretable, this specification is not restrictive: the allocation model in equation (4) can be extended to multiple discrete effort levels, allowing future work to capture graded vigilance dynamics.

A persistent challenge in vigilance research is ecological validity. Most studies rely on artificial laboratory tasks with constrained stimuli, limiting the generalizability to real-world environments (Taylor-Phillips et al., 2024). We address this gap by validating our model on naturalistic data, showing that structural estimation of attention and reward is feasible outside the lab. Another ongoing question in vigilance research concerns the design and evaluation of interventions and countermeasures, for example, breaks, incentives, or environmental modifications aimed at sustaining attention (McGough & Mayhorn, 2023; Ross et al., 2014; Waldfogle et al., 2021). Our structural model enables researchers to perform counterfactual analyses to examine how altering incentives or environmental factors might improve vigilance.

Although our empirical application centers on MLB umpire decisions, the framework generalizes to other critical domains with sufficient observational data. In radiology, performance-based vigilance measures, such as detection accuracy (Taylor-Phillips et al., 2024), serve a role analogous to umpire call accuracy. A rich literature identifies contextual variables that shape attention allocation, including characteristics of lesion such as size, contrast, visibility, and location; the prevalence of abnormalities; expertise and training; and cognitive biases such as satisfaction of search (Berbaum et al., 1990; Kundel & Nodine, 1975; Wolfe et al., 2007). These dimensions provide analogs to our “game context” features. System-level factors (Krupinski, 2010), such as display resolution and image quality, affect detection performance, much as the speed and location of a pitch affect the difficulty of umpire judgment. Our model can be similarly extended to airport security screening, where vigilance is measured by the detection of prohibited items (Meuter & Lacherez, 2016). Across these domains, our model provides a generalizable structure for inferring latent attentional states and evaluating how environmental contexts and task costs shape vigilance, helping to bridge laboratory-based vigilance theory with real-world, safety-critical applications.

Latent Context Capacity

The number of latent contexts humans can represent remains an open question in cognitive psychology (Heald et al., 2023). While some models assume that this number is known in advance (Heald et al., 2018), such an assumption can be unrealistic in naturalistic settings where context boundaries must be learned. Inference models based on latent cause theory allow for a theoretically unbounded number of contexts but are limited in practice by memory and computational constraints (Gershman et al., 2010).

In our work, we consider several candidate values for the number of hidden contexts and evaluate them using model selection criteria (such as AIC and BIC). While these techniques offer principled guidance, they do not guarantee recovery of the true underlying capacity, which is likely dynamic and individual-dependent, particularly in complex, noisy behavioral data. Nonetheless, our model captures meaningful variation in behavior and supports useful predictions or interventions. Future work could explore more flexible approaches that infer the number of latent contexts from the data.

Conclusion

This paper advances theoretical understanding of vigilance failure by introducing a Bayesian model that formalizes how attention is dynamically allocated in response to inferred context. Integrating principles from vigilance theory, decision making under uncertainty, and probabilistic inference, the model provides a principled account of how individuals regulate attentional effort in sustained attention environments. Applying this framework to MLB umpire decisions reveals that attention allocation is context sensitive and systematically influenced by situational factors (e.g., pitch count) and social cues (e.g., pitcher status), highlighting the interplay between perceived task-criticality and reputational bias. These findings refine existing models of executive control by demonstrating how latent contextual beliefs shape attentional effort in real-world settings. Future research may build on this foundation by exploring how internal states such as fatigue, cognitive load, or emotional valence interact with contextual beliefs to guide sustained attention in high-stakes decision making. Future models could also examine how attention adapts to real-time feedback. Recent advances in scalable MCMC for coupled hidden Markov models (Touloupou et al., 2020) provide an extension to our EM-based inference section, enabling a richer characterization of uncertainty.

Key Points

• A Bayesian model is developed to infer how decision makers allocate attention, adapting dynamically to perceived context in vigilance tasks.

• The model estimates latent contexts from observable task features, capturing how individuals perceive situational factors. A modified Expectation-Maximization (EM) algorithm is introduced to estimate attention allocation policies based on objective measures of task performance.

• Analysis of MLB umpire decisions reveals that attention allocation varies with game context, with umpires exhibiting increased focus in high-stakes situations.

• The model enables counterfactual predictions, allowing for the evaluation of decision making under hypothetical scenarios and different contextual influences.

Supplemental Material

Supplemental Material - A Structural Model of Attentional Effort Dynamics: Evidence From a Naturalistic Discrimination Task

Supplemental Material for A Structural Model of Attentional Effort Dynamics: Evidence From a Naturalistic Discrimination Task by Lekhapriya Dheeraj Kashyap, Zhide Wang, Yanling Chang, Alfredo Garcia in Human Factors.