Abstract
As robots increasingly integrate into human teams for critical domains, understanding the team dynamics within these multi-human-robot teams (mHRTs) becomes essential. While previous work has explored trust in human-robot interactions, there still needs to be a gap in examining how trust evolves and propagates within mHRTs. This study investigated team trust networks using neural measures in mHRTs under varying robot reliability conditions. A total of 23 teams completed search and rescue tasks in a virtual environment, with one member being an autonomous robot navigator (reliable/unreliable). Results indicate that while unreliable robot performance led to decreased trust in the robot, human dyad trust did not falter. Unreliable robot teammates weakened IBS in regions associated with individuals’ comprehension of others’ beliefs, intentions, and social judgments. These findings highlight emergent shifts in team processes under robot uncertainties that were captured using neural synchrony.
Objectives
With the advancements in AI, the utilization of robots in various domains has become increasingly prevalent, spanning from service roles in restaurants and hospitals to critical functions in search and rescue (SAR) operations (Mehta et al., 2020; Murphy, 2014). The operational environment of the robot is now shifting from a controlled environment with one-to-one interaction with one human to a real-world environment with multiple humans. Particularly in the realm of SAR, robots are integrated into human teams, creating intricate team structures to enhance operational effectiveness (Murphy, 2004). The collaboration between humans and robots capitalizes on their complementary skills, with the robot(s) of-ten assuming pivotal roles in hazardous or inaccessible environments and the humans performing the high-level control and decision-making (Murphy, 2014).
Central to the success of such collaborations is the establishment of trust networks between human and autonomy agents. In traditional human teams, intra-team trust has been identified as a key determinant of overall performance (de Jong et al., 2015). There is a growing recognition of the importance of understanding trust dynamics within multi- human-robot teams (mHRTs). Particularly in critical applications, such as SAR, unstructured adversarial environments can negatively impact the performance of an autonomy agent, and can impact team dynamics through (dis)trusting human behaviors (Feitosa et al., 2020; Honig & Oron-Gilad, 2018; Hopko & Mehta, 2022). While previous studies have explored trust dynamics in human-robot interactions, particularly in high-risk environments (R. Liu et al., 2022; McNeese et al., 2021; Robinette et al., 2016), there remains a gap in comprehensively examining how trust evolves and prop- agates within mHRTs.
Trust measurements, commonly conducted through surveys, have been a source of continuous scientific explorations (Yadav et al., 2022). Beyond surveys, individual and team behavioral measures have also offered insights into team trust dynamics (McNeese et al., 2021). However, inferring team trust through behavioral measures in naturalistic settings can be challenging, potentially disruptive, and may not account for emergent behaviors. As such, measurement systems that facilitate a continuous assessment of team trust dynamics are needed. While bio-behavioral methods, such as eye- tracking, electrocardiogram, and vocal measures, have shown potential to assess trust (Kazi et al., 2021), team cognition in- sights from these measurement systems are limited.
Collaborative activities of teamwork, as opposed to task work, require a high degree of mental and physical synchronization among team members, to the extent that synchrony underpins performance levels in activities (Hemakom et al., 2017). Hyperscanning, which involves imaging multiple brains simultaneously, offers a quantitative method to study teaming and interpersonal behaviors (Czeszumski et al., 2020). It has shed light on neural dynamics in human-human interactions, especially in scenarios requiring joint attention, communication, and cooperation (Nguyen et al., 2021). In human-robot collaboration, hyper- scanning holds promise for understanding the dynamics of the human teammates (Howell-Munson et al., 2022). For example, Yadav et al. (2023) reported higher inter-brain synchrony (IBS) in the left and right temporal parietal junction (TPJ) between the human dyads when a third member in a 3-member SAR team was replaced with a robot with comparable performance, indicating enhanced social cognition, cooperation, and social interaction (Doricchi, 2022; N. Liu et al., 2016; Pan et al., 2017). Despite lower team trust and trust in the robot, the heightened synchrony correlated with enhanced team performance, underscoring the usefulness of neural measures in grasping the juxtaposition of team dynamics. However, whether these neural measures are sensitive to changing robot performance in mHRTs remains understudied. The present study aims to broaden the examination of the relationships between IBS and mHRT dynamics (i.e., team trust networks and team performances) under robot uncertainties.
Approach
A total of 23 teams (26 M, 20 F) completed SAR tasks in a custom-designed virtual emergency environment. The task utilized a multiplayer game-like paradigm where the participants could control their virtual avatars using a keyboard and mouse (Figure 1). The team consisted of three distinct roles: a mission specialist (human), a safety officer (human), and a navigator (robot controlled using the Wizard of Oz approach). Before the start of the experiment, the dyads performed a familiarization team-based task to acquaint themselves with each other. The team was tasked to perform a search and log the victims in the virtual environment while monitoring the noxious fumes and could request navigation suggestions from the robot based on its thermal sensors. We manipulated robot performance, and each team underwent two experimental conditions: reliable multi-human-robot teams (HRR) and unreliable multi-human-robot teams (HRU). Each condition consisted of five 3-min long trials. The robot failed twice within the five trials in the HRU condition, with one of the three failure modes happening at random (stopping against a wall, repeating the same direction suggestion, announcing “system malfunction” and stopping). The human dyads could freely communicate verbally with each other and the robot. Neural data (collected using functional nearinfrared spectroscopy), subjective responses of team and individual trust metrics, fatigue levels, and mental effort, and objective mission performance (total distance traveled, number of victims located, and wayfinding score; [Shi et al., 2021]) were collected. We utilized paired t-tests (or Wilcoxon signed-rank tests depending on the normality of the data) to assess the impact of robot performance on team performance, and trust. Further, a Pearson correlation analysis was performed to uncover the relationships between these measures and neural synchrony.
Experimental setup. The experimental setup consisted of a custom designed virtual task environment simulation an emergency situation. The environment consisted of a building on fire with victims placed throughout the environment. The search and rescue team consisted of three members with distinct roles: mission specialist (H1), safety officer (H2), and a navigator (robot). The experiment design manipulated robot performance (high, low) and measures such as performance, subjective perception, and neural synchrony were captured.
Findings
Team trust (p = .019) and trust in the navigator (p < .001) decreased in the HRU compared to the HRR condition (Figure 2a–c). However, trust between the human dyads did not falter (p = .123). As shown in Figure 3a and b, higher fatigue was perceived (p = .003) in the HRU than the HRR condition, with no changes in mental workload (p = .249). Despite poorer robot performance, the teams covered more distance (p < .001) and located more victims (p < .001) in the HRU than the HRR condition (Figure 2d and e). However, the wayfinding score, a measure of efficient SAR, was not different between conditions (p = .056). Inter-brain synchrony (IBS) was higher during HRR condition in the l/r TPJ compared to HRU condition (p = 0.023 & p = 0.024; Figures 2g, h and 4). Mean and standard deviation for measured variables are shown in Table 1.
Measure captured in the reliable (HRR) and the unreliable (HRU) condition. Top row: Trust metrics. Middle row: Performance measures. Bottom row: Inter-brain synchrony measures. (Note: “ns” denotes not significant (p > .05), “*” indicates significance (.01 < p < .05), “**” denotes high significance (.001 < p < .01), and “***” signifies high significance (p < .001): (a) Trust in robot navigator, (b) Dyad trust, (c) Trust in the team, (d) Distance traveled by human dyad, (e) Number of victims identified by the team, (f) Wayfinding score of the team, (g) Inter-brain synchrony in channel 12 (PFC), and (h) Inter-brain synchrony in channel 16 (rTPJ).
Fatigue and mental workload in the reliable (HRR) and the unreliable (HRU) conditions. Human dyads perceived significantly more fatigue (p = .003) in the HRU condition compared to HRR condition. However, mental workload did not change with robot performance: (a) Perceived fatigue and (b) Mental workload.
Inter-brain synchrony (only channels with significant difference shown) in the reliable (HRR) and the unreliable (HRU) conditions. The connection between channel 12 (lTPJ) in the dyad is shown with a blue curve while channel 16 (rTPJ) is shown with orange curve. Further, the inter-brain synchrony values for each channel is shown in the two conditions (HRR and HRU). A significant difference was observed for these regions with a lower IBS value in the HRU condition compared to the HRR condition (p = .023 & p = .024).
Mean and Standard Deviations of the Measured Variables.
Further, Table 2 illustrates the significant correlations observed between team trust, behaviors, and IBS measures. Key findings are highlighted here: (a) IBS in the PFC region is positively correlated with team performance measures and trust in dyads. (b) IBS of rTPJ regions are negatively correlated with team performance (wayfinding score and victims identified), trust in the human dyads, and trust in navigator, but positively correlated with team trust. (c) IBS in lTPJ regions are positively correlated with performance measures (distance and wayfinding score). (d) An increase in mental workload reduces IBS in the l/r TPJ.
Correlation Between the Independent Measures Collected in the Study.
Note: Nav. trust represent the trust in the navigator. WF score represents wayfinding score. # victims represents number of victims located. Mental W. represent mental workload. Analysis revealed that IBS in PFC channel six positively correlated with performance: victims located (r = .518, p < .001), wayfinding score (r = .367, p = .006), and distance traveled (r = .302, p = .006). However, channel 17 (rTPJ) showed negative correlations with wayfinding score (r = −.247, p = .022), victims identified (r = −.386, p = .030), trust in navigator (r = −.114) and and dyad trist (r = −.013). While team trust was positively correlated with IBS in ch 17 (r = .157) and ch 12 (r = .033). Mental workload negatively correlated with channels 11 (lTPJ, r = −.276, p = .036) and 16 (rTPJ, r = −.202, p = .010). Performance metrics (wayfinding score [r = −.504, p = .001], victims identified [r = −.494, p = .018]) correlated negatively with fatigue.
Each cell represents Pearson correlation value with the significance indicated by asterisks (“*” indicates significance [.01 < p < .05], “**” denotes high significance [.001 < p < .01], and “***” signifies high significance [p < .001]).
Takeaways
Our findings reveal a pronounced decline in neural synchrony between the human dyads within the TPJ regions under robot unreliability. The TPJ region plays a pivotal role in facilitating individuals’ comprehension of others’ beliefs, intentions, and social judgments (Eloy et al., 2022; Schurz et al., 2017; Sebastian et al., 2011). With an unreliable robot teammate, the synchrony in the r/l TPJ regions in the brain, which are responsible for mental state attribution, attention reorienting (Decety & Lamm, 2007; Saxe & Wexler, 2005), and mentalizing (Samson et al., 2004), is weakened. Interestingly, this resulted in better team performance. Cooperative behaviors are shown to enhance IBS in the PFC (Czeszumski et al., 2022), an observation that we found, which likely explains its positive relationship with higher trust in dyads but not team trust, and thereby explaining the performance gain amidst robot uncertainty. That the teams showed higher synchrony in PFC and lower synchrony in TPJ regions indicated a shift toward cooperative interactions under robot uncertainty.
In unreliable conditions, teams exhibited emergent cooperative behaviors by minimizing reliance on the robot (Oswald et al., 2022). Generally, each team member has a specific role to follow, and the teams frequently pause to communicate with the navigator. When the robot navigation commands were not helpful, the human dyads covered more ground and found more victims to make up for the navigator’s shortcomings. Fatigue increased, which resulted in higher synchrony in the PFC, but also lowered synchrony in the TPJ, indicating cooperative over collaborative team dynamics (Kozar, 2010).
We highlighted emergent shifts in team processes under robot uncertainties that were captured using neural synchrony, which explained changes in team trust and performance out- comes.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
