Sage Journals: Discover world-class research

Abstract

Background

The number of accidents due to distracted pedestrian is on the rise and many governments and institutions are enacting public policies which restrict texting while walking. However, pedestrians do more than just texting when they use their mobile devices on the go.

Objective

Exploring pedestrian multitasking, this paper aims to examine the effects of mobile device task type on pedestrian performance outcomes.

Method

We performed two studies in lab simulations where 78 participants were asked to perform different tasks on a mobile device (playing a game, reading, writing an email, texting one person, group texting) while performing a pedestrian visual discrimination task while either standing or walking on a treadmill. Behavioral performance as well as neurophysiological data are collected.

Results

Results show that compared to a no-phone control, multitasking with any of the tasks on a mobile device leads to poor performance on a pedestrian visual discrimination task. Playing a game is the most cognitively demanding task and leads to the greatest performance degradation.

Conclusion

Our studies show that multitasking with a mobile device has the potential to negatively impact pedestrian safety, regardless of task type. However, the impacts of different mobile device tasks are not all equivalent. More research is needed to tease out the different effects of these various tasks and to design mobile applications which effectively and safely capture pedestrians’ attention.

Application

Public policy, infrastructure, and smart technologies can be used to mitigate the negative effects of mobile multitasking. A more thorough understanding of mobile device task-specific factors at play can help tailor these counter-measures to better aid distracted pedestrians.

Keywords

attentional processes distractions dual task mobile devices pedestrians

INTRODUCTION

Mobile devices are ubiquitous. They are omnipresent in all contexts such as outdoor walking or navigating in a public place such as shopping malls, subways, and streets. Having mobile devices constantly in hand leads to mobile multitasking where a pedestrian is engaged in a task performed on the mobile device while being vigilant to the physical environment surrounding them. For both pedestrians and drivers, distractions were the leading cause of injuries in road accidents in certain jurisdictions (Société de l’assurance automobile du Québec, 2017). It is estimated that about one in five pedestrians will be looking at their mobile devices while crossing an intersection (Hatfield & Murphy, 2007; Ropaka et al., 2020; Société de l’assurance automobile du Québec, 2017). The use of a cell phone also influences walking speed and visual traffic analysis before crossing (Hatfield & Murphy, 2007; Nasar et al., 2008). The use of a mobile phone was associated with a 3-fold increase in the likelihood of injury in the following minute (Ren et al., 2021). The use of a mobile device while crossing the street inhibits safe behavior and is a danger to road users in the same way as texting while driving.

Mobile device use by pedestrians—termed mobile multitasking—requires users to repeatedly switch their attention from the mobile device to the pedestrian environment and back (Nasar et al., 2008). When a person alternates between tasks in such a way, there tends to be a performance cost to the switch for both the mobile device activity and the environment monitoring activity, sometimes leading to missing an important element of the environment and causing injury or loss of life (Hodgetts & Jones, 2006). Allport et al. (1994) attribute task-switching costs to the proactive interference of the previous task over the present task. Thus, when a participant switches from Task A to Task B, the A task-set remains activated in the brain and interferes momentarily with the demands of task B (Allport et al., 1994). In addition, interference can occur when both tasks rely on similar cognitive resources (Wickens, 2008) such as the visual cognitive processing required by both the mobile device and pedestrian tasks in mobile multitasking.

In addition, the use of mobile devices has drastically changed in recent years and it is no longer safe to assume that a person looking at their screen is simply texting (Bovonsunthonchai et al., 2020). The increasing number of mobile device users in the global market has led to an enormous increase in the number of applications that consumers use on their mobile devices. In a $6.3 trillion market in 2021, the Apple App Store had nearly 4.3 million applications, while Google Play had more than 2.9 million applications and this number is expected to keep increasing. A survey reports that American mobile users interact with a variety of apps on their smartphones, with social media, email, browsing the Internet, and games being the most popular apps (AudienceProject, 2019).

While any type of application can lead to mobile multitasking, the demands of the task and type of attention required of the user by each application are different and could affect their ability to multitask. For example, multitasking appears to impact reading speed, but not reading comprehension, while it seems to affect quality, but not efficiency when it comes to writing. It also appears that different brain regions might be involved in task-switching depending on the nature of the switch required (Kim et al., 2012). We can also look to the meta-analysis presented by Horrey and Wickens on mobile multitasking while driving (Horrey & Wickens, 2006). They find that conversational tasks performed on a mobile phone tend to come with a greater cost to driving abilities as opposed to information processing tasks. They posit that conversations are more emotionally engaging, and thus, capture more of the attention of the driver. From these articles, we can infer that mobile multitasking will be differently affected based on the nature of the application being used on the smartphone.

In previous research, we used a task-switching paradigm to study the attentional and behavioral effects of using one’s smartphone while walking (Courtemanche et al., 2019). The results of the experiment showed that the probability of responding correctly was significantly lower when participants texted while walking. Also, electroencephalography (EEG) recordings showed that task-switch costs were higher when participants switched from texting to identifying the walker’s direction than vice-versa. As with our research, texting while walking is the main paradigm studied when it comes to mobile device use in pedestrians (Lopresti-Goodman et al., 2012; Strubhar et al., 2015; Thompson et al., 2013), and while these paradigms are a crucial first step, their results may not be generalizable to all mobile multitasking contexts.

Few studies have examined tasks other than texting in a mobile multitasking context. Cha et al. looked at texting and gaming’s impacts on walking and found a small impact of distraction of gait quality (Cha et al., 2015), while Schwebel et al. found that pedestrians texting or listening to music were more likely than those speaking on the phone to be struck by a car in a virtual pedestrian environment (Schwebel et al., 2012). Considering how quickly the app industry is evolving, for example with the record breaking enthusiasm for the game Pokemon Go in 2018 and the rise of location-based games since then, it is becoming more important than ever to understand how other phone activity affects the risks associated with mobile multitasking. Thus, the purpose of this paper is to examine mobile multitasking, comparing the effects of various mobile device tasks on pedestrian performance in a pedestrian–pedestrian encounter, a more common and slower collision, allowing for a better comparison of the conditions.

METHODS

Experimental Procedure

As presented by Banducci et al. (2016), pedestrian behavior can be segmented into multiple interrelated sub-tasks. For example, street-crossing can be divided into approach (walking toward the street), preparation (glancing in multiple directions and assessing the movement of other individuals and vehicles), and physically walking across the street, and each portion of behavior can be affected differently by mobile device distractions. To represent a variety of these sub-tasks, we examined mobile multitasking during multiple pedestrian-type tasks, namely preparation (i.e., assessing the movement of others while standing immobile) as well as physical movement (i.e., assessing the movement of others while walking). The experiment was designed to recreate a situation representative of meeting a walker on the sidewalk; however, it took place in a laboratory room suitable for the experiment. To represent pedestrian tasks, participants stood or walked on a treadmill and responded to an intermittent visual discrimination task representing discrimination of the orientation of a pedestrian walking towards the participant. At the same time, participants had to perform secondary tasks on their mobile device. Upon hearing an alert, participants were instructed to disregard the mobile device, look up at a screen in front of them, and complete the pedestrian visual discrimination task. They were then free to reengage with the mobile device. In study 1, this was performed while standing immobile, while in study 2, this was performed while walking on a treadmill (see Figure 1). The two studies were conducted in two different laboratory environments, and while we attempted to replicate the environment as closely as possible, it is preferable to not directly compare the two studies.

Figure 1.

Laboratory setup. Participant is standing on a treadmill, using her smartphone and wearing an EEG cap. The biological motion stimulus for the pedestrian visual discrimination task is visible on the screen in front of her.

Pedestrian Visual Discrimination Task

For the pedestrian task, a dynamic point-light walker representation of a walking human form composed of 15 black dots was used as a biological motion stimulus (see Figure 2). It represents an ecologically valid stimulus in the context of texting while walking, as pedestrians are often surrounded by other pedestrians. Recognizing other pedestrians and correctly inferring different motion characteristics such as direction or speed is essential for safety. Using a dual-task paradigm, Thornton et al. (2002) found that performance on the point walker’s directional identification is strongly effected by divided attention. It was therefore an appropriate task for comparing effects of various mobile device tasks on pedestrian performance.

Figure 2.

Dynamic point-light walker stimulus, turned slightly left and slightly right.

The dynamic walker figure was depicted by dots representing the head, shoulders, hips, elbows, wrists, knees, and ankles. The dots were presented on a white background with the figure walking toward the participant with a small angle deviation angle (see Figure 2). Based on a suggested target of 80% accuracy rate (Legault et al., 2012), pretests were used to determine that an angle of 3.0° (or −3.0°) was appropriate. The walker figure (Troje, 2008) had a height of 1.80 m and was displayed centered on the screen at a distance of 2 m from participants, giving a 25° visual angle.

Two speakers were located in front of participants to play a 1000 millisecond (ms) auditory stimulus cue with a random interval of +/− 500 ms before the presentation of the walker figure. The point-light walker figure was then displayed for 1000 ms with a resolution of 1280 × 1024 pixels using a projector (ViewSonic, US), positioned at 2 m from the wall. Participants were instructed to verbally identify the walker’s direction by answering “left” or “right” according to the side they perceived the walker would pass them. A new walker figure was presented approximately every 17 seconds (+/− 500 ms).

Mobile Device Tasks

Participants were instructed that while they were on the treadmill performing the visual discrimination task, they would also be performing a variety of tasks on a smartphone. No explicit instructions were provided regarding task prioritization; however, the pedestrian tasks were presented first and as ongoing throughout the experiment session. The smartphone tasks selected are representative of those commonly performed when using mobile devices: reading a passage from a book, writing an email, playing a game, and texting. In study 1, participants engaged in four mobile device tasks: writing an email (email), reading a document (Reading), playing a game (Gaming), and texting in a group conversation (GrText). There was an addition control condition in which no mobile device was used (Control), resulting in five conditions in total for study 1. In study 2, we wanted to further explore the two mobile device tasks that were the most difficult for participants in study 1 (namely, texting and playing a game). Study 2 consisted of three mobile multitasking tasks (texting one-on-one or OneText, texting in a group conversation or GrText, and playing a game or Gaming) as well as a no-phone control condition (Control), resulting in four conditions for study 2. Following Courtemanche et al. (2019), all texting conditions were conducted with confederates from the research team that responded actively to keep the conversation going. Also, in line with Courtemanche et al. (2019), the treadmill speed was set to 0.36 m/s. Confederates were provided with a list of open questions to start the conversation (e.g. what movie have you seen recently) but were free to engage in any topic. All gaming conditions were performed with a Tetris game that had simple controls and instructions. Mobile device tasks were performed using an X iPhone (Apple, USA).

As we were interested in understanding the effect of mobile device task type on pedestrians, pedestrian performance was measured by accuracy rate on the visual discrimination task (% correct). Similar to previous laboratory studies of texting while walking (Courtemanche et al., 2019), no performance measures were captured for the mobile device tasks as many of these tasks are hedonic in nature (e.g., texting with a group) and performance across these tasks is not meaningfully comparable (e.g., comparing game scores to texting with a group).

The experiment was composed of multiple blocks of 22 trials, one for each mobile device task conditions plus control (study 1: 5 blocks, study 2: 4 blocks). The duration of a block was approximately 7.5 minutes. The order of the blocks followed a Latin squares approach to suppress possible learning effects, and the blocks were separated by a two-minute pause in which participants could sit on a chair. Prior to the first block, participants had a two-minute practice period to familiarize themselves with the walker stimulus. The total experiment duration was 60 minutes, and a total of 110 trials per participant were collected.

Participants

Thirty participants took part in Study 1 (ages 21–43, M = 25.6, SD = 5.9). The sample included 14 males and 16 females and most participants (70%) were students. In study 2, 48 participants took part in the experiment (ages 18–46, M = 25.5, SD = 5.5). The sample included 20 males and 28 females and most participants (73%) were students. All participants in both studies had a normal or corrected-to-normal vision and were pre-screened for glasses, epilepsy, and health, neurological and psychiatric diagnoses. Participants were required to have owned and used an iPhone for at least 6 months over the last 5 years. This research complied with the American Psychological Association Code of Ethics and was approved by the ethics review board at the lead author’s university. All participants provided written informed consent before participation and were given a gift certificate as compensation upon experiment completion (40$ in study 1, 50$ in study 2).

Electroencephalography Acquisition and Analysis

Electroencephalography was recorded at 1,000 Hz during each condition to evaluate the cognitive state of the user and obtain insight into mechanisms of the observed effects. The signal was acquired with the acticap 32 pre-amplified electrodes and the BrainAmp amplifier (Brain Products, Munich, Germany) and analyzed with EEGlab (Delorme & Makeig, 2004) and the Vision Analyzer software (Brain Products, Munich, Germany). The signals were filtered offline with a 0.5–80 Hz bandpass filter and a 60 Hz notch filter. Data was downsampled to 256 Hz and artifacts were removed with the artifact subspace reconstruction algorithm (Kothe et al., 2016; Mullen et al., 2013) via the clean_rawdata plugin. This plugin from the EEGLAB software is a specialized algorithm developed for the specific purpose of removing artifacts automatically from EEG data, allowing for faster and less biased artifact removal than previous manual or semi-automatic methods. This approach has been used in previous research (Bulea et al., 2014; Perera et al., 2016) and employed as a benchmark when comparing alternative methods (Gabard-Durnam et al., 2018; Kilicarslan et al., 2016; Ojeda et al., 2019). As recommended by Chang and colleagues (Chang et al., 2020), the k parameter was set to 20, meaning data regions were removed if they exceed 20 times the standard deviation of the calibration data chosen by the algorithm. All channels were re-referenced to the common average reference.

We used the Event-related desynchronization/synchronization (ERD/ERS) transform of the BrainVision Analyzer software to obtain synchronization measures. ERD/ERS is a relative power decrease/increase of electric activity in a frequency band during a specific time window. This transform implements the method presented by Kalcher and Pfurtscheller (1995). The baseline reference interval was −1500 to −500 ms before the sound signal and the frequency bands of interest were theta 4–8 Hz and alpha 8–12 Hz as these are the frequency bands where task switching is observed. The analyses below focus on the Fz (midline frontal electrode) and Pz (midline parietal electrode). This analysis, based on the work of Wolfgang Klimesch (Klimesch, 2012), looks at the ability of the participant to inhibit the task performed on the phone and turn their attention to the visual discrimination task. Tasks that require more inhibition should be accompanied with an increase in ERS.

RESULTS

Study 1 Mobile Multitasking While Standing

Behavioral

To compare the effects of mobile device conditions on performance of the visual discrimination task, neither traditional t-tests nor an ANOVA were appropriate given that the visual discrimination task was performed repeatedly for each subject. Therefore, we used a linear regression with random intercept model, a suitable analysis given the sample size and repeated measures study design. The means were compared and the two-tailed p-values were adjusted for multiple comparisons using the Holm–Bonferroni method.

For the first study, participants performed the visual discrimination task while standing on the treadmill. Statistical analysis results of the mean pairwise comparisons can be seen in Table 1 and Figure 3. Comparing the impact of the mobile device task on performance of the visual discrimination task, we found that participants performed best in the control no-phone condition, and worst in the gaming condition. The gaming and group texting conditions were not significantly different from each other and were both significantly worse performance than the control condition. The reading and the email writing conditions did not differ significantly from control, and both led to statistically better performance than the gaming condition.

TABLE 1:

Pairwise Comparisons for Performance (Accuracy) on Pedestrian Visual Discrimination Task in Study 1

Task A	Task B	Mean Difference	SE	Adjusted 95% CI		p Value	Adjusted p Value
Task A	Task B	Mean Difference	SE	LL	UL	p Value	Adjusted p Value
Reading	Email	.040	.027	−.037	.117	.142	.427
	Gaming	.139	.027	.062	.217	<.001	<.001
	GrText	.102	.027	.025	.179	<.001	.002
	Control	−.015	.027	−.092	.062	.576	.576
Email	Gaming	.100	.027	.023	.177	<.001	.002
	GrText	.062	.027	−.015	.139	.024	.118
	Control	−.055	.027	−.132	.022	.044	.175
Gaming	GrText	−.038	.027	−.115	.039	.162	.427
Gaming	Control	−.155	.027	−.232	−.078	<.001	<.001
GrText	Control	−.117	.027	−.194	−.040	<.001	<.001

Note. Results were adjusted for multiple comparisons using the method of Holm-Bonferroni SE = standard error; CI = confidence interval; LL = lower limit; UL = upper limit.

Figure 3.

Performance (accuracy) on pedestrian visual discrimination task in Study 1. (* = p < 0.05; error bars show 1 standard deviation).

Neurophysiological

To compare cognitive activity between conditions, we analyzed the event-related synchronization (ERS) amplitude following the auditory cue, as participants need to inhibit their mobile device task to prepare for the pedestrian visual discrimination task. Increased ERS has been found in the past to be related to an increased need for inhibition in task switching (Klimesch, 2012). Frontal (Fz) locations tend to display this relationship more strongly (Sauseng et al., 2005), though parietal (Pz) regions can also show a link between ERS and inhibition (Benedek et al., 2014). Thus, the analyses focused on Fz and Pz. Using the adjusted p values, no significant differences were found between any conditions at Fz and Pz.

Study 2 Mobile Multitasking While Walking

Behavioral

For the second study, we also compared the impact of mobile device task on the performance on the pedestrian visual discrimination task, but this time, participants were walking on a treadmill. Statistical analysis results of the mean pairwise comparisons can be seen in Table 2 and Figure 4. Using the same statistical approach as in study 1, we found that participants performed best in the control no-phone condition and worst in the gaming condition. As in study 1, the gaming and group texting conditions were not significantly different from each other and both had significantly worse performance than the control condition. The one-on-one texting condition was not significantly different from the control condition, but it was significantly better than the gaming condition.

TABLE 2:

Pairwise Comparisons for Performance (Accuracy) on Pedestrian Visual Discrimination Task in Study 2

Task A	Task B	Mean Difference	SE	Adjusted 95% CI		p	Adjusted p Value
Task A	Task B	Mean Difference	SE	LL	UL	p	Adjusted p Value
Gaming	GrText	−.029	.016	−.072	.015	.080	.160
	OneText	−.043	.016	−.087	.001	.009	.037
	Control	−.080	.016	−.123	−.036	<.001	<.001
GrText	OneText	−.014	.016	−.058	.029	.379	.379
GrText	Control	−.051	.016	−.095	−.007	.002	.011
OneText	Control	−.037	.016	−.080	.007	.026	.078

Note. Results were adjusted for multiple comparisons using the method of Holm-Bonferroni. SE = standard error; CI = confidence interval; LL = lower limit; UL = upper limit.

Figure 4.

Performance (accuracy) on pedestrian visual discrimination task in Study 2. (* = p < .05; error bars show 1 standard deviation).

Neurophysiological

As in study 1, study 2 EEG results found no significant differences at electrode site Fz. At electrode site Pz, gaming had significantly less ERS than all other conditions (Pairwise comparisons to GrText: δM = −11.960, SE = 3.709, p = .008; to OneText: δM = −9.423, SE = 3.709, p = .049; to Control: δM = −15.793, SE = 3.709, p = < .001). Results were adjusted for multiple comparisons using the method of Holm–Bonferroni. None of the other conditions were significantly different from each other.

DISCUSSION

It is the nature of mobile multitasking that one needs to disengage their attention from the phone to pay attention to their surroundings. This task-switching leads to a diminished ability to assess the dangers surrounding them. This is directly related to the increase in pedestrian accidents since the introduction of mobile devices (Nasar & Troyer, 2013). It can also explain an increase in white-collar workers workplace injury rates as employees run into objects or fall down stairs while on their mobile devices at their workplace (Cohen et al., 2017). Our study investigated whether the nature of what people are doing on their mobile device influences the severity of the pedestrian problems encountered. Although much research examines the effects of one-on-one texting while walking, our behavioral results show that mobile device task type matters with certain tasks effecting visual discrimination task performance more severely, which can lead to more pedestrian-related accidents.

In our results, we can see clearly that gaming and group texting leads to worst performance, whereas tasks like reading, email and texting one-on-one did not significantly impact pedestrian visual task performance. This is in line with the findings of Horrey and Wickens (Horrey & Wickens, 2006) showing that conversational tasks come at a greater cost to task switching comparatively to information processing tasks.

Our results also align with and deepen our understanding of existing theory. Multitasking theory tells us that human cognition is a limited resource and multitasking requires effortful switches back and forth between tasks, thus reducing performance (Wickens, 2008). The literature suggests that task demands play a role in determining impacts on performance and these task demands are influenced by multiple properties—such as task engagement and the dynamic nature of the task—that influence switch costs and multitasking performance (Wickens et al. 2022).

Engagement (or cognitive tunneling) makes it difficult to switch such that when the task is more engaging, it takes more effort to disengage one’s attention and switch to the other task (Wickens & Alexander, 2009). This results in a slower switch or less attention available for the other task, thus exacerbating multitasking effects on performance. In our experiment, gaming and group texts appeared to be the most engaging tasks. Greater engagement in a gaming task could be due to greater enjoyment of the task and greater engagement in a group texting task could be due to greater extrinsic motivation to perform the task (Montgomery et al., 2004).

Previous work also suggests that tasks which are dynamic in nature and exhibit an “unstable state” (Wickens et al. 2022) require more effort to detect changes in their dynamic environment (e.g. Wickens & Alexander, 2009), resulting in negative effects on multitasking performance. While existing research focuses on the dynamic nature of the ongoing task (e.g., the pedestrian task), our work highlights that the dynamic nature of the mobile device task also matters. In our study, gaming and group texts appear to be more dynamic in nature. The gaming task was dynamic given that the pacing of play was partially determined by the game, creating an unstable and changing game environment where removing attention from the game for too long may result in lost points or losing the game entirely. Similarly, texting in a group conversation is dynamic in nature, where not responding quickly enough may result in a response that is out of order in the flow of the conversation, a response that is no longer relevant, or another person in the group “stealing” one’s answer. Other tasks such as email writing, reading, and even one-on-one texts do not have this same dynamic nature and our results show no significant differences between these mobile device tasks and the control condition.

Neurophysiological results in our work only showed a few significant results in the second study when participants were walking. In line with existing literature on the relationship between ERS and task inhibition (Klimesch, 2012), we had thought that ERS would be higher for the more engaging tasks such as gaming. Our results show the opposite however, with the gaming condition having significantly lower ERS at Pz than all other conditions. A potential explanation is that participants were so absorbed in the game that they did not actually inhibit the game effectively (as exhibited by lower ERS) to complete the pedestrian visual discrimination task (as exhibited by lower performance). At the core of this potential explanation is the idea of immersion in one’s mobile device task, termed cognitive absorption (Agarwal & Karahanna, 2000). Given cognitive absorption’s five dimensions of temporal dissociation, focused immersion, heightened enjoyment, control and curiosity, gaming mobile device tasks would likely result in higher cognitive absorption. It may be that strong absorption in one’s game makes it more difficult to inhibit this mobile device task and turn attention back to the pedestrian visual discrimination task as needed, ultimately resulting in poor pedestrian performance.

Taken together, our behavioral and neuropsychological results are important because most mobile multitasking research looks at texting while walking or, for observational studies such as Thompson and colleagues (2013), looking at your phone while walking. The content of the interaction with the phone is rarely directly examined. Our research shows that mobile device task type does matter. This has implications for both researchers of mobile multitasking as well as for mobile app developers. If the nature of a task can impact pedestrian task performance, then it is likely that the features of a given app will affect this as well.

Limitations and Future Research

Our work has several limitations, which also represent opportunities for future research. First, the difficulty of the visual discrimination task (walker angle of 3.0° or −3.0°) remained constant throughout all trials and participants’ performance on this task may have improved over time due to learning effects. Such learning effects—if present—would have made it less likely for us to detect statistically significant results and thus our current findings may understate the effects of mobile multitasking. However, future research could investigate how varying pedestrian task difficulty influences pedestrian performance during multitasking. Similarly, this study included an auditory cue to alert participants of the oncoming discrimination trial. While the cue may appear to differ from a natural pedestrian experience, the multisensory nature of this task is closely related to a situation where an oncoming pedestrian alerts someone to their presence verbally, or a car honks at a pedestrian walking where they should not. Again, future studies could employ a variety of sounds to be even more ecologically valid, while still employing a simulation for safety reasons.

Second, no performance measures were captured for the mobile device tasks. As many of the tasks were hedonic in nature (e.g., texting with a group) and it is not meaningful to compare “performance” across these tasks (e.g., comparing game scores to texting with a group), adding and analyzing such performance measures would have limited real-world implications. In hedonic tasks, task “performance” is not as meaningful as how much individuals are engaged in the task. Outcomes such as cognitive absorption—or deep immersion in the mobile device task (Agarwal & Karahanna, 2000)—may have more meaningful implications and be more comparable across tasks. To develop a deeper understanding of mobile device task types, future research could examine how an individual’s cognitive absorption in the mobile device task influences performance on the pedestrian visual discrimination task.

Third, our study sample was students with an average age of approximately 25 years old participating in a laboratory experiment, which may limit generalizability of our results. This population is one that exhibits both high mobile device use and mobile device multitasking with a variety of task types, providing ecological validity. However, older populations on average have diminished multitasking abilities (Tams et al., 2021) so the effect of mobile device task type on pedestrian performance may be even more pronounced. Additionally, while the experimental setup was selected for safety considerations, our results of the effects of mobile device task type could be replicated in other real-world contexts (e.g., Pourchon et al., 2017). The choice of Tetris for the game task may also limit the generalizability of those results. Tetris is a dynamic game in which a player bears a penalty even for short interruptions as the blocs continue to drop even if the player is inattentive. Other game mechanics with alternate subtask goal completion could have a different level of engagement on the users. Future research should replicate our work for a variety of population groups and using non-laboratory methods. Similarly, for our results to be generalizable to potential collisions with motor vehicles, future research should reproduce our findings in a vehicle-based simulation.

Finally, while neurophysiological measurements have come a long way towards recording EEG while in movement, it can be difficult to obtain high quality EEG data from moving participants. Methodological choices (e.g. pre-amplified electrodes) and analytical techniques (e.g., artifact removal) were used to partially mitigate this issue; however, the EEG results should still be interpreted with caution. Our intention is that the EEG data supplement the conclusions observed from the pedestrian behavioral performance data. The EEG data do not challenge the conclusion that mobile device task type influences pedestrian performance; however, they provide preliminary results and encourage further research on the specific attentional processes underlying our results. For example, future research could examine how cognitive absorption mediates the relationship between mobile device task type and pedestrian performance.

Implications for Practice

Our results show that the nature of what a person is doing on their phone is of great importance when attempting to reduce the risks of using your mobile phone while engaging in pedestrian related activities. Mobile device and app designers should be at the center of these discussions by developing interventions to limit the risks inherent to mobile multitasking. They are ideally positioned to affect users’ behaviors and could potentially save lives by modifying the nature of mobile device tasks by modifying their designs. For example, our results suggest that interactivity and time pressure appears to be an important contributor to diminished pedestrian performance. In the same way that some apps are now able to prevent drivers from using their mobile devices while driving, a mobile application or a wearable device could easily detect that a user is walking and simply pause remove the time pressure of the game they are playing. Alternatively, a simple wrist-flick gesture of flipping the phone around could enable users to pause the game until the phone is returned to the face-up position.

This applies especially to risk of GPS-based augmented-reality games such as Pokemon go. Pokemon go was the first widely popularized game of this kind, and since then there have been others such as Harry Potter: Wizards Unite, Zombie Run, or Orna. These games were explicitly designed with the intention of having players walking while looking at their smartphones and were strongly criticized following some serious incidents (Barbieri et al., 2017). They have since attempted to limit the risks by, for example, developing a bracelet (Pokemon go plus) that allows you to play without looking at your phone. This bracelet was however not particularly popular as it appeared to be an afterthought and did not provide the same means of interaction as the original game on the phone. Wizards unite, on the other hand, seemed to want to limit the movements of the participants in the most exciting portion of the interaction by taking a screenshot when the action begins allowing the participant to move to a safer spot to continue playing. This was not, however, explicitly stated as the intent and perhaps an explicit warning as to the dangers of playing while walking with an invitation to only start the battle once safe would have been better. Other apps have chosen to be audio based instead, such as zombie run or run an empire, with the goal of using an app to play a game and get users moving while still being safe. Location-based augmented-reality mobile games are unlikely to go away anytime soon, given their current popularity, thus more research is urgently needed to keep pedestrians safe while playing and to help developers be allies in this safety strategy.

Public policy and infrastructure are an often recommended means of influencing pedestrian behavior. Governments should be involved in curbing risky mobile multitasking behavior, through measures such as fines for people who cross the street while on their mobile devices (Russo et al., 2018) and in ground light signals (Kim et al., 2021). Such counter-measures for pedestrian distraction from mobile devices can come from four categories: infrastructure, engineering, legislation, and smart technology (Osborne et al., 2020). In terms of software engineering, we recommend that software developers acknowledge the possibility of their apps being used by pedestrians and adjust their creative processes accordingly, especially considering the following section of the IEEE/ACM code of ethics (Gotterbarn et al., 2001): “Software engineers shall act consistently with the public interest.” Our research provides a clear indication that the nature of the mobile device task can affect the safety of pedestrians, and future research should examine parameters of mobile device applications can be leveraged to improve pedestrian safety.

Footnotes

Acknowledgments

This research was partially funded by a grant from the Social Sciences and Humanities Research Council of Canada (SSHRC) to Sylvain Sénecal, Pierre-Majorique Léger and Marc Fredette, and by a grant from Natural Sciences and Engineering Research Council of Canada (NSERC) to Pierre-Majorique Léger.

Key POINTS

Compared to a no-phone control, multitasking with a mobile device leads to poor performance on a pedestrian visual discrimination task.

The type of task performed on the mobile device influences the extent to which mobile multitasking degrades pedestrian performance.

Compared to other mobile device tasks, playing a game is the most engaging and leads to the greatest degradation in pedestrian performance.

Elise Labonte-LeMoyne is a Researcher of IT at HEC Montreal and at the NSERC Industrial Research Chair in User Experience. She obtained her PhD in Exercise Science, with a specialty in Neuroscience from the faculty of Medicine at the University of Montreal in 2014.

Ann-Frances Cameron is a Professor of IT at HEC Montreal and the Holder of the Canada Research Chair in Digital Communication and Multitasking. She obtained her PhD in Management Information Systems from Queen’s University in 2007.

Sylvain Sénécal is a Professor of Marketing at HEC Montreal and the Holder of the RBC Financial Group Chair of E-commerce. He obtained his PhD in Marketing from HEC Montreal in 2003.

Marc Fredette is a Professor of Decision Sciences at HEC Montreal. He obtained his PhD in Statistics from the University of Waterloo in 2004.

Jocelyn Faubert is a Professor and Vice dean of research at the school of Optometry at the University of Montreal and. He obtained his PhD in Visual Psychophysics from Concordia University in 1991.

Franco Lepore is a Professor of Psychology at the University of Montreal and a Knight of the National Order of Quebec. He obtained his PhD in psychology from the University of Montreal in 1971.

Pierre-Majorique Léger is a Professor of IT at HEC Montreal and the Holder of the NSERC-Prompt Industrial Research Chair in User Experience. He obtained his PhD in Electrical engineering from Polytechnique Montreal in 2003.

References

Agarwal

Karahanna

(2000). Time flies when you’re having fun: Cognitive absorption and beliefs about information technology usage. MIS Quarterly, 24(4), 665–694. http://www.jstor.org/stable/3250951

Allport

D. A.

Styles

E. A.

Hsieh

(1994). Shifting intentional set: Exploring the dynamic control of tasks. In Umiltà

(Ed.), Attention and performance series. Attention and performance 15: Conscious and nonconscious information processing (pp. 421–452). The MIT Press.

AudienceProject . (2019). Insights 2019 - App & social media usage. https://www.audienceproject.com/wp-content/uploads/audienceproject_study_apps_social_media.pdf

Banducci

S. E.

Ward

Gaspar

J. G.

Schab

K. R.

Crowell

J. A.

Kaczmarski

Kramer

A. F.

(2016). The effects of cell phone and text message conversations on simulated street crossing. Human Factors: The Journal of the Human Factors and Ergonomics Society, 58, 150–162. https://doi.org/10.1177/0018720815609501

Barbieri

Vettore

Pietrantonio

Snenghi

Tredese

Bergamini

Previato

Stefanati

Gaudio

R. M.

Feltracco

(2017). Pedestrian inattention blindness while playing Pokémon Go as an emerging health-risk behavior: A case report. Journal of Medical Internet Research, 19(4), 1–9. https://doi.org/10.2196/jmir.6596

Benedek

Schickel

R. J.

Jauk

Fink

Neubauer

A. C.

(2014). Alpha power increases in right parietal cortex reflects focused internal attention. Neuropsychologia, 56(1), 393–400. https://doi.org/10.1016/j.neuropsychologia.2014.02.010

Bovonsunthonchai

Ariyaudomkit

Susilo

T. E.

Sangiamwong

Puchaphan

Chandee

Richards

(2020). The impact of different mobile phone tasks on gait behaviour in healthy young adults. Journal of Transport and Health, 19, 100920. https://doi.org/10.1016/j.jth.2020.100920

Bulea

T. C.

Prasad

Kilicarslan

Contreras-Vidal

J. L.

(2014). Sitting and standing intention can be decoded from scalp EEG recorded prior to movement execution. Frontiers in Neuroscience, 8(Nov). https://doi.org/10.3389/FNINS.2014.00376/FULL

Cha

Kim

Park

Song

(2015). Effects of mobile texting and gaming on gait with obstructions under different illumination levels. Physical Therapy Rehabilitation Science, 4(1), 32–37. https://doi.org/10.14474/ptrs.2015.4.1.32

10.

Chang

C.-Y.

Hsu

S.-H.

Pion-Tonachini

Jung

T.-P.

(2020). Evaluation of artifact subspace reconstruction for automatic artifact components removal in multi-channel EEG recordings. IEEE Transactions on Biomedical Engineering, 67(4), 1114–1121. https://doi.org/10.1109/TBME.2019.2930186

11.

Cohen

B. J.

Larue

Cohen

H. H.

(2017). Attention interrupted: cognitive distraction & workplace safety. Profession Safety, 62(11), 28–34.

12.

Courtemanche

Labonté-LeMoyne

Léger

P.-M.

Fredette

Senecal

Cameron

A.-F.

Faubert

Bellavance

(2019). Texting while walking: An expensive switch cost. Accident Analysis & Prevention, 127, 1–8. https://doi.org/10.1016/j.aap.2019.02.022

13.

Delorme

Makeig

(2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009

14.

Gabard-Durnam

L. J.

Mendez Leal

A. S.

Wilkinson

C. L.

Levin

A. R.

(2018). The harvard automated processing pipeline for electroencephalography (HAPPE): Standardized processing software for developmental and high-artifact data. Frontiers in Neuroscience, 12, 97. https://doi.org/10.3389/fnins.2018.00097

15.

Gotterbarn

Miller

Rogerson

, et al. (2001). Software Engineering Code of Ethics and Professional Practice. Science and Engineering Ethics, 7(2), 231–238. https://doi.org/10.1007/s11948-001-0044-4

16.

Hatfield

Murphy

(2007). The effects of mobile phone use on pedestrian crossing behaviour at signalised and unsignalised intersections. Accident Analysis and Prevention, 39(1), 197–205. https://doi.org/10.1016/j.aap.2006.07.001

17.

Hodgetts

H. M.

Jones

D. M.

(2006). Interruption of the Tower of London Task: Support for a Goal-Activation Approach. Journal of Experimental Psychology: General, 135(1), 103–115. https://doi.org/10.1037/0096-3445.135.1.103

18.

Horrey

W. J.

Wickens

C. D.

(2006). Examining the Impact of Cell Phone Conversations on Driving Using Meta-Analytic Techniques. Human Factors: The Journal of the Human Factors and Ergonomics Society, 48, 196–205. https://doi.org/10.1518/001872006776412135

19.

Kalcher

Pfurtscheller

(1995). Discrimination between phase-locked and non-phase-locked event-related EEG activity. Electroencephalography and Clinical Neurophysiology, 94(5), 381–384. https://doi.org/10.1016/0013-4694(95)00040-6

20.

Kilicarslan

Grossman

R. G.

Contreras-Vidal

J. L.

(2016). A robust adaptive denoising framework for real-time artifact removal in scalp EEG measurements. Journal of Neural Engineering, 13(2), 026013. https://doi.org/10.1088/1741-2560/13/2/026013

21.

Kim

Cilles

S. E.

Johnson

N. F.

Gold

B. T.

(2012). Domain general and domain preferential brain regions associated with different types of task switching: A Meta-Analysis. Human Brain Mapping, 33(1), 130–142. https://doi.org/10.1002/hbm.21199

22.

Kim

Kwon

Choi

Shin

(2021). Performance of ground-level signal detection when using a phone while walking. Accident Analysis & Prevention, 151(November 2020), 105909. https://doi.org/10.1016/j.aap.2020.105909

23.

Klimesch

(2012). Αlpha-band oscillations, attention, and controlled access to stored information. Trends in Cognitive Sciences, 16(12), 606–617. https://doi.org/10.1016/j.tics.2012.10.007

24.

Kothe

C. A. E.

Jung

T-P.

, & US Patent App. 14/895, 440, & 2016, U. (2016). Artifact removal techniques with signal reconstruction. In (Eds.), Google Patents. https://patents.google.com/patent/US20160113587A1/en

25.

Legault

Troje

N. F.

Faubert

(2012). Healthy older observers cannot use biological-motion point-light information efficiently within 4 m of themselves. i-Perception, 3(2), 104–111. https://doi.org/10.1068/i0485

26.

Lopresti-Goodman

S. M.

Rivera

Dressel

(2012). Practicing Safe Text: the Impact of Texting on Walking Behavior. Applied Cognitive Psychology, 26(4), 644–648. https://doi.org/10.1002/acp.2846

27.

Montgomery

Sharafi

Hedman

L. R.

(2004). Engaging in activities involving information technology: Dimensions, modes, and flow. Human factors, 46(2), 334–348.

28.

Mullen

Kothe

Chi

Y. M.

Ojeda

Kerth

Makeig

Cauwenberghs

Jung

T-P

(2013). Real-time modeling and 3D visualization of source dynamics and connectivity using wearable EEG. 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 3–7 July 2013, Osaka, Japan, 2184–2187. https://doi.org/10.1109/EMBC.2013.6609968

29.

Nasar

Hecht

Wener

(2008). Mobile telephones, distracted attention, and pedestrian safety. Accident Analysis & Prevention, 40(1), 69–75. https://doi.org/10.1016/j.aap.2007.04.005

30.

Nasar

J. L.

Troyer

(2013). Pedestrian injuries due to mobile phone use in public places. Accident Analysis & Prevention, 57, 91–95. https://doi.org/10.1016/j.aap.2013.03.021

31.

Ojeda

Klug

Kreutz-Delgado

Gramann

Mishra

(2019). A Bayesian framework for unifying data cleaning, source separation and imaging of electroencephalographic signals. Biorxiv.Org. https://doi.org/10.1101/559450

32.

Osborne

Horberry

Young

K. L.

(2020). Pedestrian distraction from Smartphones: An end-user perspective on current and future countermeasures. Transportation Research Part F: Traffic Psychology and Behaviour, 73, 348–361. https://doi.org/10.1016/j.trf.2020.07.007

33.

Perera

Shiratuddin

M. F.

Wong

K. W.

(2016). A review of electroencephalogram-based analysis and classification frameworks for Dyslexia. In Hirose

Ozawa

Doya

et al., (Eds.) Lecture Notes in Computer Science (Vol. 9950 LNCS, pp. 626-635). Springer Verlag. https://doi.org/10.1007/978-3-319-46681-1_74

34.

Pourchon

Léger

P.-M.

Labonté-LeMoyne

É.

Sénécal

Bellavance

Fredette

Courtemanche

(2017). Is augmented reality leading to more risky behaviors? An experiment with Pokémon Go. In Fui-Hoon

Tan

C. H.

(Eds.), HCI in Business, Government and Organizations (pp. 354-361). Springer. https://doi.org/10.1007/978-3-319-58481-2_27

35.

Ren

Chen

Xue

Yin

Peng

Liang

Feng

Wang

(2021). Road Injuries Associated With Cellular Phone Use While Walking or Riding a Bicycle or an Electric Bicycle: A Case-Crossover Study. American Journal of Epidemiology, 190(1), 37–43. https://doi.org/10.1093/aje/kwaa164

36.

Ropaka

Nikolaou

Yannis

(2020). Investigation of traffic and safety behavior of pedestrians while texting or web-surfing. Traffic Injury Prevention, 21(6), 389–394. https://doi.org/10.1080/15389588.2020.1770741

37.

Russo

B. J.

James

Aguilar

C. Y.

Smaglik

E. J.

(2018). Pedestrian behavior at signalized intersection crosswalks: Observational study of factors associated with distracted walking, pedestrian violations, and walking speed. Transportation Research Record: Journal of the Transportation Research Board, 2672(35), 1–12. https://doi.org/10.1177/0361198118759949

38.

Sauseng

Klimesch

Doppelmayr

Pecherstorfer

Freunberger

Hanslmayr

(2005). EEG alpha synchronization and functional coupling during top-down processing in a working memory task. Human Brain Mapping, 26(2), 148–155. https://doi.org/10.1002/hbm.20150

39.

Schwebel

D. C.

Stavrinos

Byington

K. W.

Davis

O’Neal

E. E.

de Jong

(2012). Distraction and pedestrian safety: How talking on the phone, texting, and listening to music impact crossing the street. Accident Analysis & Prevention, 45(2), 266–271. https://doi.org/10.1016/j.aap.2011.07.011

40.

Société de l’assurance automobile du Québec . (2017). 2016 Road safety record.

41.

Strubhar

A. J.

Peterson

M. L.

Aschwege

Ganske

Kelley

Schulte

(2015). The effect of text messaging on reactive balance and the temporal and spatial characteristics of gait. Gait & Posture, 42(4), 580–583. https://doi.org/10.1016/j.gaitpost.2015.09.007

42.

Tams

Grover

Thatcher

Ahuja

(2021). Grappling with modern technology: interruptions mediated by mobile devices impact older workers disproportionately (pp. 1–21). Information Systems and e-Business Management.

43.

Thompson

L. L.

Rivara

F. P.

Ayyagari

R. C.

Ebel

B. E.

(2013). Impact of social and technological distraction on pedestrian crossing behaviour: An observational study. Injury Prevention, 19(4), 232–237. https://doi.org/10.1136/injuryprev-2012-040601

44.

Thornton

I. M.

Rensink

R. A.

Shiffrar

(2002). Active versus passive processing of biological motion. Perception, 31(7), 837–853. https://doi.org/10.1068/p3072

45.

Tran

Carrillo

Subrahmanyam

(2013). Effects of online multitasking on reading comprehension of expository text. Cyberpsychology: Journal of Psychosocial Research on Cyberspace , 7(3), Article 2. https://doi.org/10.5817/CP2013-3-2

46.

Troje

(2008). Retrieving information from human movement patterns. In Shipley

T. F.

Zacks

T. M.

(Eds.), Understanding Events: From Perception to Action (pp. 308–334). Oxford University Press.

47.

Wickens

C. D.

(2008). Multiple Resources and Mental Workload. Human Factors: The Journal of the Human Factors and Ergonomics Society, 50(3), 449–455. https://doi.org/10.1518/001872008X288394

48.

Wickens

C. D.

Helton

W. S.

Hollands

J. G.

Banbury

(2022). Engineering psychology and human performance (5th ed.). Routledge. https://www.taylorfrancis.com/books/9781003177616

49.

Wickens

C. D.

Alexander

A. L.

(2009). Attentional tunneling and task management in synthetic vision displays. The International Journal of Aviation Psychology, 19(2), 182–199. https://doi.org/10.1080/10508410902766549

What’s that on Your Phone? Effects of Mobile Device Task Type on Pedestrian Performance

Abstract

Background

Objective

Method

Results

Conclusion

Application

Keywords

INTRODUCTION

METHODS

Experimental Procedure

Pedestrian Visual Discrimination Task

Mobile Device Tasks

Participants

Electroencephalography Acquisition and Analysis

RESULTS

Study 1 Mobile Multitasking While Standing

Behavioral

Neurophysiological

Study 2 Mobile Multitasking While Walking

Behavioral

Neurophysiological

DISCUSSION

Limitations and Future Research

Implications for Practice

Footnotes

Acknowledgments

Key POINTS

References