Abstract
The influence of gaze cues on target prioritisation (reaction times [RTs]) and movement execution (movement trajectories) differs based on the ability of the human gaze cue model to manually interact with the targets. Whereas gaze cues consistently impacted RTs, movement trajectories may only be affected when the hands of the human model had the potential to interact with the target. However, the perceived ability to interact with the targets was confounded by the proximity between the model’s hands and the targets. The current study explored if the influence of gaze cues on movement trajectories is shaped by the model’s potential to access and interact with the targets using their hands or simply the proximity of the hands. A centrally presented human model randomly gazed towards one of two peripheral target locations. Participants executed aiming movements to targets that non-predictively appeared at one location at a stimulus onset asynchrony of 100, 350, or 850 ms. In Experiment 1, the model’s hands could not directly access the targets as each was holding a tray. In Experiment 2, the hands had direct access to the targets, but their palms-downwards orientation and wrist-flexed posture rendered efficiently interacting with the targets unlikely. Although RTs showed a facilitation effect of the gaze cue in both experiments, changes in movement trajectories were only observed when the model had direct access to the target (Experiment 2). The results of the current study suggest that the gaze model’s direct hand access is necessary for the social gaze cues to influence movement execution.
Non-verbal social cues, such as the direction of eye gaze or of a pointed finger, provide important information about the observed individual’s intentions and objects of interest. These non-verbal social cues can be used by the observer to plan their own actions to either coordinate or avoid collisions with the observed individual. Understanding how social cues, such as gaze direction, influence motor planning and execution is crucial for advancing the knowledge of social cognition and perceptual-motor interactions. The present study investigates the impact of gaze cues on the initiation and execution of manual aiming movements, focusing on whether the observed human model’s perceived ability to act on an object or mere proximity of the model’s effectors influences how gaze cues affect movement execution. Specifically, the current study addresses a significant gap in the literature by exploring how the perceived ability of a gaze model to interact with targets affects the planning and execution of movement by examining the movement’s onset and trajectory. Although previous studies have demonstrated that social cues can alter the salience of gazed-at locations or objects and influence response times (e.g., Friesen & Kingstone, 1998; Frischen, Bayliss, & Tipper, 2007), the specific mechanisms by which gaze cues affect movement planning and execution remain underexplored and available evidence indicates some unexpected differences (Wang et al., 2024). The present research thus provides new insights into how these cues interact with the motor system, particularly in distinguishing between the effects of cue proximity versus the potential for action.
The tendency to follow another person’s gaze or other social directional cues (e.g., pointing) is ubiquitous and can occur in a manner that is largely outside of conscious control (Friesen et al., 2004; Friesen & Kingstone, 1998; Langton & Bruce, 2000). This tendency for such social cues to increase the salience of a given location, direction, or object can be readily observed in, for example, social pranks (e.g., “look at the sky” prank), sports (e.g., head fakes in basketball), and subtle nonchalant looks to discretely point out something interesting to friends in a public space. Behavioural research has supported these observations by showing that social cues can result in the preferential processing or salience of external sensory input, a process termed the orienting of attention (Frischen, Bayliss, & Tipper, 2007; McKay et al., 2021). In an empirical setting, the involuntary orienting of attention by non-social and social cues has broadly been studied using spatial cueing tasks in which a non-predictive visual cue is directed towards a possible target location prior to the appearance of the target (Posner, 1980). This task was designed to test the prediction that, if the visual cue reorients attention towards the location or direction of the cue in an involuntary manner, then the processing of stimulus information at the location of the cue will be processed more efficiently and accurately than information at other locations. The results of numerous studies have supported this prediction—responses for targets at cued locations have shorter reaction times (RTs) and/or are more accurate than responses at “uncued” locations. For example, Friesen and Kingstone (1998) presented participants with a central face whose gaze was suddenly directed to a left or right location before a target randomly appeared at the gazed-at (cued) or non-gazed-at (uncued) location. RTs for targets at cued locations were shorter than for targets at uncued locations, suggesting that the central gaze cue oriented the attention and increased the efficiency of information processing at the gazed-at location over the other location. Critically, because the visual cue is not predictive of the target location, the participant did not have a voluntary drive to orient attention to the gazed-at cued location. As a result, the differences in processing stimulus information at a cued location relative to stimulus information at an uncued location are attributed to involuntarily activated mechanisms that affect prioritisation or salience of the location.
Interestingly, social and non-social cues that drive involuntary shifts of attention seem to activate different prioritisation mechanisms. These different mechanisms have been revealed through examinations of the effects of the temporal separation between the cue and target onset (stimulus onset asynchrony; SOA) on accuracy and/or RTs of responses to the target. Following social cues, such as eye gaze and a pointed hand, RTs typically tend to be shorter for the cued targets than for the uncued targets across all SOAs (ranging from 0 to 1,000 ms), indicating persistently facilitated responses towards gazed-at targets (with some rare exceptions; Friesen & Kingstone, 1998; Frischen, Smilek, et al., 2007). In contrast, for non-social cues, such as the sudden onset or brightening of a potential target location, the facilitation effect at shorter SOAs (e.g., <200 ms) is replaced by an inhibitory effect at longer SOAs (e.g., >300 ms) wherein RTs are longer for targets at the cued location than for targets at uncued locations. This latter effect, a phenomenon termed inhibition of return or IOR, is thought to emerge because the initial facilitatory effect is eventually outweighed by an inhibitory effect, which has developed to hinder the return of attention to a previously attended location (Klein, 2000; Posner & Cohen, 1984; Posner et al., 1985). The emergence of IOR following non-social cues, but not social cues, is an interesting distinction that might be indicative of different processing streams for these types of stimuli (Pitcher & Ungerleider, 2021).
In a study designed to further investigate potential differences in the attentional and coupled motor system mechanisms activated by social cues, Yoxon et al. (2019) examined the effect of non-predictive social eye gaze and hand-pointing cues on the characteristics of manual aiming movements. Yoxon et al. employed a manual aiming task because detailed analyses of the temporal and spatial characteristics of aiming movements can provide insight into both the prioritisation of information presented at specific locations (via RTs) and the impacts of mechanisms associated with this prioritisation on movement planning and execution (via trajectory deviations; Gallivan et al., 2018; Welsh & Weeks, 2010). In separate blocks of trials, participants were presented with a disembodied head on gaze cue trials or an isolated pointing hand (separated from an arm and body) on hand trials. The gaze and hand stimuli were presented between two open squares that indicated the potential target locations. At the beginning of each trial, participants placed their index finger in a “home” circle at the bottom of the screen and the centrally presented model’s gaze or the pointing hand was directed towards the participant. After a 1,000 ms foreperiod, the eye gaze or the pointing finger would shift to be randomly directed towards the target on the left or right. Following a random SOA ranging from 100 to 2,400 ms after the motion of the cue, one of the target squares would turn solid and the participants were asked to point to and touch the solid target location as quickly as they could. The direction of the social cue did not predict the location of the target. The range of SOAs was chosen because, although social eye gaze cues tend to produce facilitation effects across a large range of SOAs (see Frischen, Bayliss, & Tipper, 2007, for review), non-social cues lead to facilitation effects with SOAs of less than 200 ms (with peak facilitation between 100 and 150 ms), a transition from facilitation to inhibition effects with SOAs of 200–300 ms, and inhibitory effects with SOAs of greater than 300 ms (with stable inhibition emerging at SOAs between 600 and 1,000 ms; see Klein, 2000; Posner & Cohen, 1984; Welsh et al., 2013).
Yoxon et al. (2019) recorded the position of the tip of participants’ index finger and analysed RTs and movement trajectories to evaluate the effect of the cue and SOA on target prioritisation and movement planning. RTs were defined as the temporal separation between target onset and movement initiation, and movement trajectory was operationalised as the initial movement angle (IMA), or the absolute angle between the principal axis (the midline that evenly separates the two sides of the display from the home position) and the finger’s position at 20% of the movement. The authors found that both gaze and pointing cues elicited a persistent facilitation effect in RTs: RTs for cued target trials were consistently shorter than RTs on uncued target trials. Conversely, gaze cues did not affect IMA, whereas the pointing hand cue elicited a facilitation effect in IMA, where movements on uncued trials deviated towards the cued location following pointing cues only. Deviations towards the cued location on uncued trials are thought to indicate that the cue activates a response code to the cued target location in the motor system, and this simultaneously presents a non-target response code that competes and interferes with the response code to the target (Welsh & Elliott, 2004; Welsh et al., 1999). Based on this pattern of findings, the authors suggested that the eye gaze and hand-pointing cues were processed differently—both social eye and hand cues can generate shifts of attention and prioritisation, but only hand cues subsequently activated responses leading to manual aiming movements of the hand to the cues.
Wang et al. (2024) further investigated the potential for gaze cues to influence movement execution. One potential reason that the pointing, but not gaze, cues may have influenced the planning and execution of a manual pointing task is the similarity between the effector and the cue might have contributed to the salience of the cue to the perception and action system that governed the movement. Recall that in Yoxon et al. (2019), participants were executing manual aiming movements following gaze and pointing cue stimuli that were presented in a disembodied head and hand, respectively. Hence, there were similarities between the effector (participant’s hand) in the pointing condition (model’s pointing hand) but not in the gaze condition (model’s head). In light of this hypothesis, Wang et al. suggested that if the gaze model was given the potential to interact with the target with his hands, or act-ability, then the gaze cue may also affect movement execution. Across two experiments, an adult male gaze model with a full upper body was centrally presented in a similar setup as in Yoxon et al. In Experiment 1, the model had his arms extended to the sides and hands placed palms upwards directly underneath the potential target locations, whereas the arms of the model were crossed in front of his chest in Experiment 2. The arms and upper body of the model were visible in both experiments, but the model had a high potential to interact with the possible targets in Experiment 1 and a low potential to interact with the possible targets in Experiment 2. Furthermore, instead of only measuring IMA as in Yoxon et al., Wang et al. compared the entire movement trajectory for the cued and uncued trials.
RTs results replicated previous findings for both experiments: There was a consistent facilitation effect at shorter SOAs in RTs, but no inhibitory effect at longer SOAs across both Experiments 1 and 2. For the trajectory analysis, a facilitation effect was observed at 350 ms SOA (movements on uncued trials deviated towards the cued location), whereas an inhibitory effect emerged at 850 ms SOA (movements on uncued trials deviated away from the cue). Interestingly, these facilitation and inhibition deviation effects only emerged when the model had a high potential to act on the target (Experiment 1), but neither a facilitation nor inhibitory effect was observed when the model had a low potential to act on the target (Experiment 2). Based on these findings, the authors suggested that it was not the mere presence of the model’s arms and hands, but the perceived ability of the gaze model to directly interact with the targets that provided the context sufficient for the gaze cue to elicit orienting of attention and the response activation to generate movement deviations.
This overall set of findings across this series of studies could be attributed to a common neural component, the medial temporal area (V5/MT), for processing dynamic social cues (Pitcher & Ungerleider, 2021; Stephenson et al., 2021) and motor planning and execution (Gilaie-Dotan, 2016; Whitney et al., 2007). Wang and colleagues (2024) provided initial evidence that the action-relevant characteristics of the cueing model and the model’s gaze direction interact, subsequently influencing the observer’s motor planning and execution. However, it may be the proximity between the model’s hands and the targets (or low-level convergence of visual information near the target locations), rather than the model’s potential for action, that leads to the activation of the motor system and the observed changes in movement trajectories. To determine if hand proximity or the potential for action leads to the generation of trajectory deviations, the current study used a setup that was similar to Wang et al. (2024) and presented the same cueing model with arms that were extended to the sides and close to the target locations. In Experiment 1, the palms of the model’s hands were facing upwards while holding a tray, whereas the palms were facing downwards with wrists flexed in Experiment 2. Thus, the model’s hands had similar proximity to the targets in both studies (as in Experiment 1 of Wang et al., 2024), but had different potentials to directly access and interact with the targets.
It was predicted that facilitation effects in RTs would be observed in both experiments because, thus far, facilitatory gaze cueing effects have been observed in all experiments, regardless of the context surrounding the model (Wang et al., 2024; Yoxon et al., 2019). Hence, the experimental predictions outlined here will focus on the pattern of trajectory deviations. If mere proximity between the hands and the targets, or low-level convergence of visual information near the target location, is sufficient to lead to motor system activation following a gaze cue, then a pattern of facilitatory and inhibitory trajectory deviations similar to that observed in Experiment 1 of Wang et al. (2024) should be observed in both Experiments 1 and 2. However, if the potential for the model to directly access and interact with the targets using the hands is needed to activate the motor system, then trajectory deviations should be observed in Experiment 2 (where the hands can directly access the object) but not in Experiment 1 (where the tray would prevent direct access to the objects because the placement of the trays in the hands of the model in Experiment 1 may present a barrier). Finally, if activation of the motor system following a gaze cue is dependent on the ability of the model to directly and efficiently interact with the target locations, then no trajectory deviations should emerge in either experiment. Trajectory deviations might not emerge in each experiment because: (1) in Experiment 1, the tray may prevent the hands from efficiently interacting with the targets, and (2) in Experiment 2, the model cannot efficiently interact with the object with the back of their hand.
Experiment 1
Methods
Participants
Twenty adults (13 females and 7 males, mean age = 24.35 years, SD = 4.58) participated in this study in exchange for monetary compensation. All participants had normal or corrected-to-normal vision and were right-handed. All procedures were approved by and were consistent with the University of Toronto’s Research Ethics Board. Participants provided full and informed consent before participation. One participant’s data were discarded due to technical difficulties during spatial calibration.
Stimuli and apparatus
The stimuli were presented on a 24-inch Dell monitor with a 1920 × 1080 resolution and 60 Hz refresh rate. The monitor was slanted at an approximately 32° angle from a horizontal tabletop and was facing the participant. This setup ensured the participant’s comfort when performing the pointing movements. The experiment was implemented in PsychoPy (Peirce et al., 2019). As shown in Figure 1a, a filled blue circle with a 1.5 cm radius was placed at the bottom of the screen as the home position. The home position was placed along the screen’s vertical midline and 13 cm below its centre. An image of a young adult male, used as the cue model, was placed above the home position. The model had his arms to the side, hands opened, and palms facing upwards while holding a tray in each hand (Figure 1a). These images were the exact images used in Experiment 1 of Wang et al. (2024) except for the addition of the trays. Two unfilled squares with a side length of 2 cm were placed above the model’s hands to indicate potential target locations. To ensure the targets were above the trays, the target locations were rendered 13.5 cm from the screen’s horizontal midline and 5.83 cm from its vertical midline. Finally, note that the target locations were aligned with the eyes of the model.
Time course of a single trial for (a) Experiment 1 and (b) Experiment 2. At the beginning of each trial, the participants rested their right index finger on the home position (blue circle). Two open squares were placed over the model’s hands to indicate potential target locations. After a 1,000 ms foreperiod, the model shifted his gaze to one of the potential target locations. Following one of the stimulus onset asynchronies (SOA; 100, 350, or 850 ms), one of the target squares would turn solid, which was the signal for the participants to point to the solid target as quickly as they could.
The participant’s movement trajectory was recorded using an opto-electric motion tracking system (Optotrak, Northern Digital Inc., Waterloo, ON, Canada). An infrared-emitting diode (IRED) was taped onto the fingernail of the participant’s right index finger. The three-dimensional (3D) coordinates were recorded at a 250 Hz sampling frequency. Data collection was controlled asynchronously in PsychoPy through a custom Optotrak interface in Python so that the sampling frequency for movement data is not restricted to the screen’s refresh rate.
Procedure and design
At the beginning of each session, the experimenter informed the participant of the experiment’s scope and procedure. The participants would only sign the informed consent form and proceed with the experiment if they did not have any questions about the task and risks. The participant sat in front of a table with a slanted monitor. The experimenter used medical tape to attach the IRED to the fingernail of the participant’s right index finger. Because the motion capture device was calibrated relative to the horizontal tabletop, spatial calibration was performed so that later the slanted 3D movement trajectories could be transformed to a horizontal plane for data reduction. During spatial calibration, the participant was instructed to sequentially point to the four corners of the screen with their right index finger following a beeping sound. The end positions of their finger were then recorded.
Figure 1a shows the timeline of a single trial. At the beginning of each trial, the participant placed their right index finger on the home position. Following a 1,000 ms foreperiod, the model’s gaze would shift left or right towards one of the target locations. His gaze direction would remain in that direction for the remainder of the trial. Then, after a random variable SOA (100, 350, or 850 ms), one of the target placeholders would turn solid, indicating target onset. The participant was instructed to point to the target as quickly as they could. The model’s gaze direction did not predict the location of the target, and participants were informed of their non-predictive nature at the beginning of the experiment. The participant was also instructed to fixate on the male model’s gaze up until the appearance of the target. The participant’s movement was recorded for 1,500 ms using Optotrak with recording beginning immediately after the target onset. Finally, the participant was instructed to hold their finger at the target location until the end of the 1,500 ms data collection period, at which point an instruction screen would appear to prompt the participant to return to the home position.
For the experimental design, there were 2 gaze directions (left/right), 2 target locations (left/right), and 3 SOAs (100, 350, and 850 ms), resulting in 12 unique trials. The collection of the unique trials was treated as a block and was repeated 16 times, resulting in a total of 192 trials. Trials within each block were presented in different random orders. The first block was considered as training and was not included in the analysis. The entire study took approximately 30 min to complete.
Data reduction
Data reduction was performed using the Trajectory Analysis Toolkit for Human Movements (TAT-HUM) in Python (Wang & Welsh, 2024). First, four movement endpoints from the spatial calibration trials were extracted as reference points for each participant. The reference points were passed through the compute_transformation_3d() function from TAT-HUM to derive the rotation matrix that transforms movement trajectories on a slanted surface to a horizontal surface. After the transformation, the primary movement axis was determined as the direction that extends away from the participant’s body (in depth), whereas the secondary axis was in the lateral (left/right) direction. For each trial, raw movement trajectories were smoothed using a low-pass Butterworth filter with a 250 Hz sampling frequency and a 10 Hz cutoff frequency for each axis to remove high-frequency noise. Then, movement velocity was calculated for each axis using a central difference method. Movement initiation and termination were determined using the resultant velocity (Pythagorean) along the primary and secondary movement axes with a threshold of 50 mm/s. Because of the coincidence between the target and movement recording onset, RT was defined as the time interval from the start of the movement data collection until movement onset, whereas MT was the time interval between movement onset and termination.
For the trajectory analysis, coordinates along each axis within the movement boundaries were first parameterised using a third-order B-spline, mapping the spatial displacement values to their corresponding timestamps:
This procedure created a representation of the movement trajectory as a function of time that enabled continuous resampling of the movement coordinates using arbitrary timestamps within the movement boundaries (Gallivan & Chapman, 2014; Ramsay & Silverman, 2005). Using the B-spline representation, 100 evenly spaced timestamps within the movement boundaries were used to derive their corresponding coordinates for every trial. An equal number of coordinate samples enabled spatial averaging across trials, which was applied to each unique combination of participant, SOA, target location, and cue location. If the cue had a facilitatory effect on movement execution due to the activation of a response to the cue, then movement trajectories in the uncued trials would deviate towards the previously cued location. This deviation towards the location of the cue on uncued trials would be revealed by a more centrally directed movement on uncued compared with a (relatively more direct) movement on cued trials (Figure 2a and b). An inhibitory effect from the cue would be revealed by movements that deviate away from the cued location on uncued target trials. This deviation away from the location of the cue on uncued trials would be revealed by a less centrally directed movement on uncued compared with a (relatively more direct) movement on cued trials (Figure 2c and d). To capture this effect, the lateral deviation (along the x-axis) between the cued and uncued trials was evaluated. The areas under the curves corresponding to the cued and uncued trials were derived by integrating the x-coordinate’s B-spline parameterisation and the area of the uncued trials was subtracted from that of the cued trials. If there was a facilitation effect, the area corresponding to the cued trials should be greater than that of the uncued, resulting in a positive trajectory area. Similarly, an inhibitory effect would result in a negative trajectory area. Finally, the trajectory area was divided into 5 segments based on the 100 timestamps and their corresponding indices (1–20, . . ., 81–100) and the total area was derived for each segment for the subsequent statistical analysis.
Examples of (a and b) a facilitatory and (c and d) an inhibitory effect are demonstrated through the difference in mean movement trajectories and their corresponding lateral displacement (x-axis) between the cued (green, dashed curves) and uncued (red, solid curves) trials.
Power analysis
An a priori power analysis was performed to determine the appropriate sample size. The most theoretically relevant goal of the present experiment is to explore the interaction effect of SOA (3 levels: 100, 350, and 850 ms) and trajectory segments (5 levels: 1–20, . . ., 81–100) on the trajectory area. An R package, Webpower (Zhang & Yuan, 2018), was used for power analysis for repeated-measures analysis of variance (ANOVA) focusing on interactions between two within-subject factors. Based on Experiment 1 of Wang et al. (2024), the significant interaction had an effect size of
Statistical analysis
A repeated-measures ANOVA was used on RTs and MTs. Given two gaze directions and two target locations, the target could either be cued (the gaze cue was in the direction of the target) or uncued (the gaze cue was in the opposite direction of the target). Therefore, each ANOVA contained two within-subject factors, SOA (3 levels: 100, 350, and 850 ms) and Congruency (2 levels: cued and uncued). ANOVA was performed using R’s afex package (Singmann et al., 2015). Greenhouse–Geisser corrections were applied to factors that failed to satisfy the sphericity assumption (indicated by decimal values in the reported degrees of freedom). Post hoc pairwise comparisons with Tukey’s corrections were used to examine the source of significant effects. For trajectory areas, a repeated-measures ANOVA was performed with two within-subject factors, including SOA (3 levels: 100, 350, and 850 ms) and Trajectory Segment (1–20, . . ., 80–100). Then, a series of pre-planned two-tailed one-sample t-tests were performed for each segment (5 levels) at each SOA (3 levels), comparing the mean areas with 0. A trajectory area value significantly greater than 0 indicates a facilitatory effect in movement execution, whereas a value significantly smaller than 0 suggests an inhibitory effect.
Results and discussion
Reaction time
For RTs, the repeated-measures ANOVA revealed that both main effects were significant, SOA:
Mean (a) reaction time, (b) movement time, and (c) area between the averaged cued and uncued movement trajectories for Experiment 1.
Movement time
A repeated-measures ANOVA on MT showed a significant main effect of SOA, F (2, 36) = 5.36,
Trajectory area
A repeated-measures ANOVA on the trajectory area showed that neither the main effect of SOA,
The results of the current experiment formed an interesting contrast to those from Experiment 1 of Wang et al. (2024). Specifically, the facilitatory and inhibitory effects that the gaze model had on movement trajectory when the hands had access to the target locations (Experiment 1, Wang et al., 2024) were not present with the addition of a pair of trays that blocked the model’s hand access to the targets (Experiment 1, current paper), even though the hands were in close proximity to the targets. This finding confirmed that hand proximity alone is not the necessary condition for the gaze model to elicit a facilitatory/inhibitory effect on movement execution. More importantly, this finding also suggests that the hand’s direct access to the target may be crucial in eliciting the gaze cueing effect on movement trajectories.
Experiment 2
Experiment 2 was designed to provide additional insights into the extent to which direct hand access would affect movement execution. Specifically, having the model’s hands placed underneath the target, the gaze model has direct access to the targets. However, because his hands are formed in an uncommon pose (facing downwards in a flexed position), the gaze model may not be perceived to have the intention to interact with the targets. Therefore, if the gaze model’s perceived act-ability is sufficient to elicit the gaze cueing effect on movement execution, then, unlike Experiment 1, a facilitatory and/or inhibitory effect should be observed in movement trajectories in Experiment 2. Conversely, if the model’s perceived act-ability is also mediated by his perceived intention to interact with the target in a common way, then, like Experiment 1, Experiment 2 would also yield a null effect of gaze cueing on movement trajectories.
Methods
Participants
Twenty adults (10 females and 10 males, mean age = 20.05 years, SD = 1.05) participated in this experiment either as volunteers or in exchange for monetary compensation. All participants had normal or corrected-to-normal vision and were right-handed. All procedures were approved by and were consistent with the standards of the University of Toronto’s Research Ethics Board. Participants provided full and informed consent prior to participation.
Stimuli and apparatus
Experiment 2 used the same stimuli and apparatus as in Experiment 1, except that the same cueing model had his arms lifted on the sides and his palms facing downwards in a flexed pose (Figure 1b). This posture, while unusual, still offered the hands unmediated access to the targets. To keep the targets above the hands in this slightly different location of the hands in this posture, the target locations were changed to 4.5 cm from the screen’s horizontal midline and 14 cm above the screen’s centre.
Procedure and design
Experiment 2 used the same experimental procedure and design as in Experiment 1. There were 16 blocks of 12 unique trials (3 SOAs × 2 target locations × 2 gaze directions) for a total of 192 trials. The first block was considered as practice and was removed from the analysis.
Data reduction and statistical analysis
Experiment 2 used the same analysis protocol as in Experiment 1.
Results and discussion
Reaction time
A repeated-measures ANOVA showed a significant main effect of SOA, F (2, 38) = 82.80,
Mean (a) reaction time, (b) movement time, and (c) area between the averaged cued and uncued movement trajectories for Experiment 2.
Movement time
A repeated-measures ANOVA on MTs did not show any significant main effects, SOA: F (2,38) = 1.77,
Trajectory area
For trajectory areas, a repeated-measures ANOVA showed a significant interaction between SOA and Trajectory Segment, F (3.05), 57.93) = 2.79, p < 0.05,
Overall, trajectory analysis revealed a facilitation effect at the 350 ms SOA, but an inhibitory effect at the 850 ms SOA. The pattern of results from Experiment 2 of the current paper replicated the findings of Experiment 1 in Wang et al. (2024). Comparing the gaze models between the two experiments, the two sets of hands have similar direct access to the target locations while the primary difference lies in the posture of the model’s hands and potential interactions with the target locations. In the previous Wang et al. experiment, the model’s hands were facing palms upwards with fingers open in a pose that could be construed either as grabbing the targets should the model lift his arms, or as catching the targets should they fall. In contrast, in the current experiment, the model’s hands were facing palms-downwards with the wrists flexed. This posture is rather unusual for a person to take if they want to interact with any objects. From one perspective, this unusual posture may be perceived by the participant as the model displaying a lack of intention to interact with the targets. From a practical perspective, the model could easily extend his wrists from the flexed position, which would invariably allow him to hit the targets with his hands. In other words, the model has direct access to the targets as well as demonstrates the potential to interact with the targets, but may do so in an unusual manner. Nevertheless, this context for the model still yielded a facilitatory and inhibitory effect on the movement trajectories, suggesting direct access to the objects, and not mere hand proximity or perceived intention of the gaze model, may be the key feature that sets the conditions needed to drive motor system activation following gaze cues.
General discussion
The present study examined the effect of social gaze cues and the gaze model’s perceived ability to interact with targets on target prioritisation and movement execution. Authors of a previous study (Wang et al., 2024) argued that social gaze cues could only elicit a facilitatory and an inhibitory effect in movement execution when the model was perceived to be able to interact with the target. The current study aimed to test this claim by exploring whether it was the mere proximity between the hands and the targets that created the cueing effect in movement execution, or whether the model had to be able to directly interact with the targets with his hands. To this end, participants executed manual aiming movements to targets in a gaze cueing paradigm in which the upper limbs of the gaze cue model could interact with the targets or not. Specifically, the gaze cue model had his arms placed out to the sides with hands either facing palms upwards while holding a tray (i.e., hands are proximal to the target without direct access; Experiment 1) or facing palms-downwards and in a flexed posture (i.e., hands are proximal to the target with direct access; Experiment 2). For both experiments, the temporal characteristics of the movements indicated the presence of a facilitation effect of the gaze cues in target prioritisation that resulted in shorter RTs for cued targets than uncued targets. However, only Experiment 2, not Experiment 1, showed the cueing effect in movement execution, including a facilitation effect at 350 ms SOA and an inhibitory effect at 850 ms SOA. Combined with results from the previous study by Wang et al. (2024), the current study confirmed that the mere proximity between the gaze cue model’s hands and the target was not sufficient to affect movement planning and execution. Instead, it seems essential for the gaze model to have direct access to the targets with his hands to elicit the cueing effect for movement planning and execution.
The most consistent finding across the series of studies conducted thus far is the relatively persistent facilitation effect that eye gaze cues have on RTs. Although not all facilitatory cueing effects were statistically significant in the present study, the directing of eye gaze towards a potential target location seems to increase the efficiency of processing of stimulus information at that location when key press (Friesen et al., 2004; Friesen & Kingstone, 1998) or aiming movements are required (Wang et al., 2024; Yoxon et al., 2019). The persistence of these facilitation effects across longer SOAs stands in contrast to the inhibitory effects from non-social spatial cues at SOAs of longer than 300 ms, whether keypress (e.g., Posner & Cohen, 1984; Posner et al., 1985) or manual aiming (e.g., Neyedli & Welsh, 2012) responses are required. It is unclear why the inhibitory mechanisms activated by non-social cues are not activated by social gaze cues, but it might be related to the pathways that process the different forms of information (Pitcher & Ungerleider, 2021) or the meaning of the information derived from stimuli. Future research should be dedicated to increasing the understanding of these differences. Nonetheless, this persistent facilitation effect in RTs from the gaze cues was replicated in the present study.
Although inhibitory effects were not observed in RTs, they were observed in movement trajectories. Consistent with the findings in Experiment 1 of Wang et al. (2024) and other studies using non-social sudden onset cues (e.g., Lee, 1999; Neyedli & Welsh, 2012), movement trajectories on uncued trials deviated towards the cued location at short SOAs and away from the cued location at longer SOAs. This pattern of deviations suggests that, initially, the shifting of attention to the gazed-at location activated a response for the motor system to interact with that location. At short SOAs, this non-target response representation to the gazed-at location was present and in an excited state when the target appeared, which subsequently activated another response. At the time of target appearance, the non-target response code was still in an excited state when the movement was initiated and, thus, the response to the uncued target contained characteristics of both target and non-target responses, causing the movement to deviate towards the cued location on uncued target trials. At the longer SOA, enough time passed before the target was presented that the non-target response to the cue was inhibited to a below-baseline level. Because the overall response activation is now coded for a response opposite to the direction of the cue, the movement deviated away from the initially cued location (Howard & Tipper, 1997; Welsh & Elliott, 2004; Welsh & Weeks, 2010). Again, it is interesting to note that evidence of inhibition consistent with IOR was present in the trajectory deviations, but not the RTs even though the measures were derived from the same set of stimuli. The relative independence between RT (decision-making) and movement trajectory (action specification) is consistent with prior literature on perceptuomotor decision-making (Cisek, 2006; Cisek & Kalaska, 2005; Manzone & Welsh, 2023; Requin et al., 1973; Welsh & Elliott, 2004; Welsh et al., 2013). RT (the time from stimulus onset to movement initiation) is primarily influenced by the initial planning stages of the movement that involve the perceptual and cognitive recognition of target and non-target information, the selection of a goal, and the preparation of the motor plan to achieve that goal. Movement trajectories, especially early in the movement, are thought to be determined by the summation of the activation levels of target and non-target response codes. The decision to initiate movement (indicated by RT) can occur at any point after stimulus onset, either before or after the target has been selected from non-targets. Errors in the initial movement trajectory (either by the presence of an active or inhibited response code to a non-target) can be modified online in response to changing sensory feedback or updated internal representations of the target/non-target information. In the current context, the additional social component of the stimulus may have modulated the processing of the non-target responses potentially leading to altered motor output (see the following paragraph), which is subsequently manifested in the movement trajectory.
Turning to the main research question, the combined results of the current and previous work indicate that the human model having direct access to and the potential to act on the target locations is key to creating the conditions needed for eye gaze cues to activate responses in the motor system. This conclusion can be drawn because: (1) trajectory deviations were not present when the arms were visible, but folded across the chest of the model (Experiment 2, Wang et al., 2024); (2) trajectory deviations were not present when the hands were near the target locations, but separated from the target locations by a tray (Experiment 1, current study); and (3) trajectory deviations were present when the hands were near and could have direct access to the target (Experiment 1, Wang et al., 2024; Experiment 2, current study). Overall, it is not the mere presence or proximity of the hands to the target, but direct access and potential for interaction of the hands that shapes the conditions needed for motor system activation from the eye gaze cue.
Although the combined data offer an understanding of the conditions leading to target prioritisation and motor system activation following gaze cues, the reasons and the underlying neural pathways that contribute to this effect can only be speculated upon at this point. Previously, the effect of the model’s act-ability and social gaze cue on movement execution was attributed to the interaction between the dorsal pathway that controls motor planning and execution (Gilaie-Dotan, 2016; Whitney et al., 2007) and the dynamic social pathway that processes social cues (Pitcher & Ungerleider, 2021; Stephenson et al., 2021) via the shared neural processing component V5/MT. For the present Experiment 1, the addition of a pair of trays on the hands of the model could be interpreted as a form of tool-mediated interaction. Tool use entails two separate knowledge domains, the action and function domains, that correspond to the activation of dorsal and ventral pathways, respectively (Q. Chen et al., 2016; Fridman et al., 2006; Tobia & Madan, 2017). While action knowledge is associated with information on how a tool can be physically manipulated to complete a certain task (hence the dorsal activation), function knowledge is related to the conceptual understanding of the tool’s intended purpose and usage (hence the ventral activation). In this context, the lack of effect of gaze cues on movement trajectory may imply that the trays were perceived functionally resulting in heightened ventral activation. Such ventral activation may have weakened the interaction between the dorsal and dynamic social pathways in the gaze cueing paradigm, resulting in the gaze cues’ lack of activation in the motor system with the resulting absence of an effect on movement trajectory. In other words, social gaze cues require the gaze model’s direct hand access to elicit facilitatory and inhibitory effects on movement execution.
Finally, it should also be acknowledged that the difference between Experiments 1 and 2 could also be attributed to other extraneous factors. For instance, the presence of the trays in Experiment 1 could be perceived as distractors or barriers, potentially influencing participants’ movement execution. This account is supported by the observation that Experiment 1 yielded longer MTs compared with Experiment 2 (mean difference ≈ 60 ms), despite similar RTs between the two Experiments (mean difference ≈ 25 ms). The longer MTs suggest that the trays might have introduced additional cognitive or motor processing demands during movement execution, possibly due to their role as distractors rather than as functional tools. Future studies should specifically investigate the effects of tool-mediated interactions on the social gaze cueing effect, focusing on how the familiarity and perceived utility of the tool may modulate both the planning and execution phases of movement. In addition, exploring variations in the tools used or incorporating more nuanced measures of attention could provide deeper insights into the mechanisms driving these effects.
In sum, the current study further demonstrated the necessity of a holistic approach when examining the social cueing effect from gaze cues. Across many studies that investigate this effect, the stimuli were almost always static images of portions of a person’s body, such as a disembodied head, a disembodied hand, or a disembodied foot (M. M. Z. Chen et al., 2020). Aside from video calls and other technologically mediated methods, social interactions commonly occur in a three-dimensional physical environment where the observers have perceptual access to the other person’s entire body. In this case, not only would the visual appearance of the model affect the socially mediated interactions, but the abundance of visual information could also impact how various social cues are perceived (Troje, 2023; Wang & Troje, 2023). In other words, while only presenting parts of a body on a screen is necessary in an empirical setting to control for extraneous factors, doing so may also neglect the synergistic effect that perceiving holistic stimuli in a naturalistic setting may have on the corresponding cognitive processes. Therefore, future studies should consider using alternative presentation methods, such as virtual reality, that could ensure both rigorous experimental manipulations and more faithful replication of the social interaction environment.
Footnotes
Acknowledgements
We would like to thank Goran Perkic for being the gaze model for this study.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada.
