Abstract
Driver mental workload may represent a mismatch between task demands and available mental resources. If driving mental workload arises from a mismatch involving executive functions, then drivers with more executive function ability should perceive less mental workload in driving situations. To test this hypothesis, 33 participants rated the mental workload associated with 16 driving scenarios and also carried out three cognitive assessment games designed to measure executive functions (response inhibition, working memory updating, switching/shifting). We found a significant relationship between two of the executive function abilities (response inhibition, and switching/shifting, but not working memory updating) and mental workload ratings. With an increase in the age of participants, we observed lower response inhibition and higher perceived mental workload after viewing representations of driving scenarios. These results demonstrate that previous results showing higher perceived mental workload for older drivers may be, at least partly, due to decreases in executive function ability as people age.
Introduction
Driver’s mental workload is one of the major issues in transportation safety as it affects drivers’ situational awareness and reaction time to hazardous situations. While researchers agree that the construct of mental workload is meaningful, and can be measured, the way in which cognitive operations and task demands combine to create the mental workload that is experienced is still a matter of debate. One approach is to view mental workload in terms of attentional resources (e.g. Young and Stanton, 2002), and to see optimization of mental workload at some appropriate intermediate level (i.e., neither too high nor too low as a way to ensure good task performance (e.g., Wilson and Rajan, 1995). In contrast to the human factors emphasis on attentional resources, psychologists have tended to characterize cognitive ability in terms of executive functions and working memory. For instance, Miyake et al. (2000) summarized research suggesting that the three main components of executive functioning were inhibition, switching/shifting, and working memory updating.
Measurement of mental workload is challenging, and many of the methods described, and technical challenges noted, in the seminal book edited by Hancock and Meshkati (1988) remain relevant as of this writing. One instrument for assessing mental workload, which relies on subjective reports, is the NASA TLX (Hart and Staveland, 1988). Another subjective report system is RSME (Ziljstra, 1995) which can be implemented as a slider ranging from 0 to 100 with nine verbal anchors.
While many research studies have examined workload caused by secondary tasks (e.g., using in-vehicle infotainment systems, or phones), some researchers have examined the impact of individual differences on the amount of mental workload that is experienced. For instance, Cantin et al. (2009) examined the effect of age on mental workload while simulated driving. Their results showed that older (vs. younger) drivers experienced greater mental workload while driving, and this effect was greater for more complex driving contexts. A series of studies by Mizobuchi and colleagues showed that individual differences in executive functions affect driving-related performance. They found that participants with lower shifting, updating, and inhibition abilities were more impacted by secondary tasks in simulated driving (Mizobuchi et al., 2011). Higher inhibition ability was related to better performance in detection-response tasks, while higher updating ability was related to better onedimensional tracking performance (in a quasi-car following task; Mizobuchi et al., 2012), and people with higher cognitive ability were able to visually attend to the primary task more, without negatively impacting their performance on the secondary task (Mizobouchi et al., 2013). Mizobuchi et al. proposed that experienced mental workload reflects the amount of mismatch between task demands and available capacity with respect to these three main executive functions. Broadbent et al. (2023) investigated the relationship between working memory capacity and (simulated) driving performance under increased cognitive load. They found that driving infractions (ex. crashes, speeding, indicator disuse when changing lanes) under dual-task conditions were negatively correlated with participants’ working memory capacity.
In the study reported below we examine whether the executive function view of mental workload can account for observed differences in perceived mental workload across different driving ages and scenarios. In a companion paper (names omitted for blind review, in review) we examined the validity of using storyboard and video representations to assess perceived mental workload associated with driving scenarios.
Methods
Participants
Participants were recruited anonymously through the third-party platform Prolific. The following inclusion criteria were used: 1). Resides in North America and holds a legally obtained driver’s license in North America; 2). Aged between 25 and 55 years old; 3). Driving experience longer than one year; 4). Able to carry out the experiment on a laptop or PC that has access to the internet. The participant criteria were enforced by the Prolific platform and each participant performed the experiment him or herself, at home or in a location of his/her choosing.
Forty participants enrolled in the study; however, seven participants’ responses were excluded due to incomplete or invalid submissions. Therefore, data from 33 participants were analyzed in this study. The following demographic information from participants was collected: age, sex, years of driving experience, driving frequency, self-rated driving skill, and the state or province that they lived in. The distributions of demographic characteristics for participants are shown in Figure 1. The gender distribution was relatively balanced, with 19 female participants and 14 male participants. The age groups of participants were relatively diverse, with 4 participants in the 17-25 age group, 10 in the 26-35 age group, 12 in the 36-45 age group, 6 in the 46-55, and 1 in 56-65 age groups.
Experiment Participant Demographics.
Cognitive functionality measurement
We used BrainTagger games (Urakami et al., 2021, https://intro.braintagger.com/) to measure participants’ cognitive functionality. BrainTagger is a suite of Whack-amole style games, which are designed for cognitive training and assessment. BrainTagger is composed of a number of TAG-ME (Target Acquisitions Games for Measurement and Evaluation) of which we used the following three games:
TAG-Me Only (TMO), TAG-Me Again (TMA) and TAG-Me Switch (TMS). Figure 2 shows how these BrainTagger games, and corresponding executive functions, are situated with respect to Baddeley’s model of working memory. The presence of the driver in the figure illustrates the connection of the executive functions to the driving task.
Decomposition of driver’s central executive functions, working memory, and relationship with TAG-ME games.
TAG-Me Only (TMO) is designed for assessing
TAG-Me Again (TMA) is designed for assessing working memory
TAG-Me Switch (TMS) is designed as an analogue of the
Rating Mental Workload Associated with Driving scenarios
In this study author HJ created 3D driving scenarios in the Unreal Engine, focusing on the representation of medium to high-workload driving scenarios. Unreal Engine was chosen due to its advanced rendering capabilities, intuitive user interface, and extensive community support. To achieve realism, HJ implemented traffic logic, vehicle physics, and random pedestrian systems within the scenarios. The traffic logic was utilized to simulate real-world traffic conditions, such as stop signs, traffic signals, and traffic patterns. The engine’s built-in physics engine was used to simulate vehicle movements, including acceleration, braking, and turning. Furthermore, random pedestrian and cyclist systems were introduced using Unreal Engine’s artificial intelligence system to create a more lifelike driving environment. Additionally, collision detection was implemented to ensure pedestrian safety (Figure 3). After the creation of 3D driving scenarios, we recorded 16 videos in the Unreal Engine each depicting unique medium to high-workload driving scenarios. Each video was approximately 10 seconds in length and was captured using a graphically simulated high-definition camera mounted on the vehicle. To provide an alternative representation of each scenario, a storyboard was created, consisting of 3-5 keyframes from the video and accompanying text descriptions. The keyframes were chosen to highlight specific aspects of the driving scenario that were salient in terms of the task demands being generated.
Driving scenario creation with Unreal Engine simulation. a). pedestrian stochastic walking system; b). traffic flow logic; c). physics ecosystem encompassing vehicle and pedestrian dynamics; d-e).
Experimental Procedure
The high-workload videos and storyboards were presented online to the participants (who were experienced drivers) for evaluation. The research was conducted online for approximately 45-50 minutes per person. An online survey tool (Figure 4): 1). Presented the experiment consent form and obtained consent; 2). Collected basic demographic information such as driving frequency and age group (using 10-year cohorts); 3). Implemented the main study.
Pages in online rating tool. a). demographic information collection; b) TAG-ME game suites. c) driving scenario ratings.
Participants played the BrainTagger games in random order, and the two driving scenario blocks were interspersed between the three cognitive assessment games. Practice trials were provided prior to each cognitive assessment game so that participants became familiar with each game before playing it. Each of the two driving scenario blocks consisted of 8 driving scenarios (4 simulated videos and 4 storyboards). Scenarios were randomly assigned to type of representation (video vs. storyboard) within participants, so that different participants saw different combinations of scenarios as videos or storyboards. Each participant saw all 16 driving scenarios with 8 shown as videos and 8 shown as storyboards. For each scenario approximately half of the participants saw it as a storyboard and half the participants saw it as a video. When viewing each driving scenario (presented as either a video or a storyboard) participants rated the mental workload associated with the scenario using a slider (0-100, with feedback on the current number selected), consistent with the RSME method (Ziljstra, 1995).
The experimental session concluded with an open-ended question (optional) asking participants if they had any impressions or feedback about the experiment that they would like to share.
In addition to rating the amount of workload in a driving situation using a slider in the driving scenario task (Figure 4a) and playing the cognitive assessment games (Figure 4b), the participants were also asked to pick the most difficult parts of the driving situation from a list of options (Figure 4c). When they made their judgments on the video the participants could play the video as many times as they wanted but they could not go back to a previous video once it had been given a rating.
After the three cognitive assessment games were played, and the two driving scenario blocks were carried out, the participants were taken to the final feedback page, where their feedback on the videos/storyboards was collected. When the participants clicked the completion button their data was sent to the database and Prolific arranged payment of the incentive. Participants were able to withdraw from the experiment at any time by closing the browser window.
Results
In order to assess the relationship between executive functioning and perceived mental workload we obtained four game performance measures, two for TAG-ME Only (d-prime and median correct RT) and one measure each for TAG-ME again and TAG-ME Switch (d-prime, and the average number of trials per rule change, respectively). Regression analysis was then carried out with the four game performance measures as predictors and mean workload rating per participant as the criterion. The regression model was statistically significant (F[4,28] = 3.0, p=0.036), with R2=.30. As can be seen in Table 1, the strongest predictors in the fitted model were the average trials per rule change in TAG-ME Switch, and median correct RT in TAG-ME Only.
Linear regression analysis with four game performance measures as predictors and average workload per person as the criterion (N=33).
In addition to assessing the relationship between executive functioning and workload, we were also interested in how executive function ability affected “accuracy” of mental workload ratings as estimated by the deviation of each person’s rating for a scenario from the mean rating for that scenario (i.e, the “workload deviation”). We carried out a second regression analysis, again with the four game performance measures as predictors, but with the mean “workload deviation” for each participant as the criterion. This second regression model was statistically significant (F[4,28] = 2.887, p=0.041, with R2=.292. As can be seen in Table 2, the strongest predictors in the fitted model were again the average trials per rule change in TAG-ME Switch, and median correct RT in TAG-ME Only.
Linear regression analysis with four game performance measures as predictors and average workload deviation per person as the criterion (N=33).
Unsurprisingly, there was a strong positive correlation of 0.86 between age and driving experience. As illustrated in Figure 5, and consistent with the findings of Cantin et al. (2009) participants who were older and had more extensive driving experience assigned higher ratings of workload (Age effect: F(3,722) = 34.976, p<.001; Driving experience effect: F(3,722) = 29.98, p<0.01). Note that since only one participant was aged over 55, we merged that person into an over 46 group for this analysis.
Workload rating for the video and storyboard representations across different age groups.
We hypothesized that the reason why perceived mental workload increases with age is because of the decrease in executive function ability that occurs with age, but with the caveat that people tend to be more cautious with age, trading less speed for more accuracy. This tradeoff can be seen in the right panel of Figure 6, where average reaction time on the TMS switching task increased with age, allowing the older participants to actually perform better on the TMS game in terms of average number of trials needed for each rule change (left panel of Figure 6). However, this tradeoff cannot be similarly exploited in the TAG-ME Only task, where participants should not only respond quickly, but should also avoid making false alarms (hitting distractors). The middle panel of Figure 6 shows that TMO correct RT increases with age, reflecting decreased response inhibition ability. This result was not affected by a speed-accuracy tradeoff since the false alarm rate was close to zero for all age groups.
Relationship between performance in games and age group (left: average trial per rule change TMS; middle: median correct reaction time TMO; right: average reaction time TMS).
Discussion
As expected, higher executive function ability (and younger age) was related to lower perceived mental workload associated with driving scenarios. Older age was associated with both lower executive function ability (in terms of response inhibition ability) and greater perceived mental workload. While a higher average number of trials per rule in the TMS game was also related to higher perceived mental workload, it is unclear whether this is also an aging effect, since there was a speed-accuracy tradeoff in the TMS task, with older participants responding more slowly, but with fewer trials per rule on average.
We found no impact of working memory updating ability on perceived mental workload. Although Broadbent et al. (2023) found that working memory capacity was related to driving infractions in simulated driving, this occurred in a dual task setting. Thus our interpretation is that working memory ability is important when someone is performing a secondary task while driving, but not when someone is rating perceived workload in a driving scenario where there is no secondary task.
Since there was a high correlation between age and driving experience, people with greater driving experience also perceived higher mental workload in the scenarios. However, this driving experience effect is likely due to the fact that people with more driving experience tend to be older and thus we see the apparent driving experience effect on executive function ability, and perceived mental workload, as a proxy for the underlying aging effect. The present results also suggest that the working memory updating component of executive functions is not related to ratings of perceived mental workload.
We also looked at how well people can estimate mental workload associated with driving scenarios, as operationalized in terms of deviation from the mean mental workload assigned to the scenarios by all the participants. As shown in Table 2, ability to estimate mental workload (as operationalized by agreement with others) was greater for people with higher levels of response inhibition ability and switching/shifting ability.
Limitations
Driven scenarios were represented using videos and storyboards. Future research should replicate these results in a driving simulator. The sample of 33 participants used in this study is relatively small for a correlational study and thus the effects obtained were of relatively borderline significance relative to a.05 alpha level. However, the sample was balanced in terms of sex and comprised a relatively wide range of ages, and the results were consistent with the hypothesized relationships. The effect sizes obtained in the regression analyses (close to or around 30% of the variance in the criterion) were also relatively strong.
Conclusion
Our overall interpretation of the results obtained is that lower levels of inhibition and switching/shifting ability lead to higher perceived mental workload of driving scenarios. This relationship also explains why older drivers tend to report higher mental workload in driving scenarios, since executive function ability decreases with age. The present results support the notion that executive function ability influences perceived mental workload associated with driving scenarios, but further research with larger samples, and with other methodologies (e.g., studies in driving simulators) are needed to confirm the present results.