Assessing Team Performance in Complex Team-Based Command and Control Missions Through Information Sharing and Team Cohesion

Abstract

In complex command and control (C2) scenarios, effective team performance depends on the development of shared situation awareness (SA) among team members with heterogeneous expertise. Civilians were assigned heterogeneous roles in ad hoc emergency management teams responding to a fictional hurricane scenario, such that mission success would require effective sharing of their unique knowledge during a team discussion session. Whereas previously published work using this dataset found relationships between SA and self-reported team cohesion, the current work compared team decisions against a benchmark “expert” team, where each expert received all information from all roles prior to the discussion session. Results showed that greater similarity between civilian and expert team decisions, indicating more effective information sharing, was related to higher team cohesion, more updating of individual SA, and greater overlap in shared SA. Facilitating information sharing and promoting team cohesion may be valuable methods for improving team effectiveness in C2 scenarios.

Introduction

Background

In military, emergency management, and similar complex command and control (C2) missions, heterogeneous teams of experts are regularly deployed. Operating interdependently, these teams strive to accomplish a shared mission, with each team member assuming a specific role with access to role-specific information. Achieving successful team performance in these missions involves fostering a cohesive understanding of mission-relevant information held both individually and collectively; that is, developing situation awareness (SA).

SA is the understanding of an environment that allows an individual to respond appropriately, by perceiving (level 1 SA) and comprehending (level 2 SA) elements within the environment, and then projecting (level 3 SA) those elements into their anticipated future states (Endsley, 1995). Within heterogeneous teams, individual team members each develop individual SA, but the team also collectively develops shared SA where the individual SA of two or more members overlaps. Importantly, the development of SA within a C2 mission is dynamic and iterative: team members identify and share relevant information from their individual SA, the team collectively develops shared SA, and team members update their individual SA in response to new information gleaned from others (Salas et al., 1995).

The planning process establishes a foundation for the subsequent C2 functions of directing, coordinating, and managing operations, while developing SA at both individual and team levels (Riley et al., 2006). In military settings, the planning process is directed by the desired outcomes established through the commanding officer’s intent. Planning in non-military contexts (e.g., emergency management) focuses on coordinating the efforts of various agencies (e.g., fire, police, paramedics) to address a problem (e.g., a natural disaster).

In both military and non-military settings, mission analysis is the first step in plan development. Mission analysis is a collaborative activity whereby team members brainstorm, identify, and discuss information that may impact mission success. Insights from the mission analysis are used to develop one or more courses of action (COAs) to achieve the mission objectives. The goal is to ensure the team has thoroughly assessed factors affecting the mission and considered multiple possibilities, allowing them to remain flexible and react appropriately in situ, rather than focusing on an optimized, prescriptive COA (Hunter & Randall, 2024).

Previous Work

Previous research suggests relationships between team performance, trust (e.g., the extent to which team members feel they can rely on one another’s knowledge and abilities), cohesion (e.g., the extent to which team members feel like valued members of a functional team), and effective information sharing (e.g., Beal et al., 2003; Huang, 2009; Mesmer-Magnus & DeChurch, 2009). A recent paper also suggested a reciprocal relationship between cohesion and SA in co-located teams (Guastello et al., 2022). However, the nature of these relationships has not been much examined in teams completing complex C2 missions, in which knowledge is heterogeneous, in which teams may be geographically distributed, and in which there may not be a single correct COA by which to gauge team performance. It stands to reason that information sharing may be even more central to team performance in these instances.

In previously published work, we explored these relationships using a fictional hurricane scenario in which civilians assumed heterogeneous roles on an emergency management team (Chubala et al., 2023). Greater self-reported team cohesion was related to some measures of level 3 SA. Although the absence of a single “correct” COA in the scenario replicated the uncertainty and complexity inherent in C2 missions, it limited the extent to which level 3 SA and team performance could be assessed.

Current Work

The current study sought to further explore the hurricane scenario dataset, investigating relationships between information sharing, trust, cohesion, and team performance. To measure team performance, we compared teams’ responses to level 3 SA questions against the responses provided by a benchmark “expert” team. Participant team members received information pertinent to their own unique and heterogeneous role on the team, and subsequently relied on effective information sharing during a team discussion session to develop their SA. By contrast, expert team members each received all information pertinent to all three heterogeneous roles prior to a team discussion session.

To the extent that participant teams effectively shared information about their individual roles, their responses should more closely match those of the expert team (i.e., better team performance), and better team trust and team cohesion should emerge. This paper also sought to contextualize the quantitative results with preliminary qualitative analyses, exploring transcripts from participant teams’ discussions and written responses to long-answer questions about COA decisions.

Method

Participants

The experiment was conducted with approval from the Dalhousie University Research Ethics Board (REB#2021-5413). Participants were recruited through social media posts and word of mouth within the authors’ networks and scheduled to attend a discussion session in teams of three. A total of 60 participants in 20 teams participated in the full experiment; one team was excluded from analyses due to a procedural error, leaving a total of 57 participants in 19 teams. The expert team was comprised of three of the paper’s authors (CC, LD, and HN).

Materials

To simulate team processes in a complex C2 environment, a scenario was developed for an imagined hurricane event that may provoke an outbreak of cholera in Haiti, requiring team members to consider several different risk factors as the event unfolded. The scenario consisted of a series of five increasingly specific information packages incorporating both written information and graphical aids, which led participants through the timeline of a hurricane event. Importantly, participants received some unique information specific to the heterogeneous roles they were assigned within their teams: either a humanitarian aid expert, a public health expert, or a migration and social services expert. Members of the expert team each received all information from all three roles. A more complete description of the task materials is given in Chubala et al. (2023).

Measures

Custom questionnaires were developed to assess all three levels of participants’ SA over the course of the experiment. Level 1 and level 2 SA questions were graded, as described in Chubala et al. (2023). However, level 3 SA questions could not be graded, as there was no single correct projected outcome of the scenario and thus no objectively correct responses.

Of particular relevance to this paper, participants assigned risk ratings to each of the 10 regions of Haiti at three different time points, indicating their beliefs about the risk of cholera outbreak following the hurricane event. Pre-Discussion and Mid-Discussion ratings were completed individually, whereas Post-Discussion ratings were completed collectively by each team after their discussion session. Risk ratings in this task represent level 3 SA because they required participants to perceive and comprehend risk information as the hurricane event unfolded, and to project the relative risks in each region into the future.

Each set of risk ratings was taken as a ten-dimensional vector, with one dimension for each of the 10 regions of Haiti. Vector cosine similarity was used to compare risk ratings both: (1) between individual members within a team (within-team similarity, a proxy for shared SA), and (2) between participant teams and the expert team (team performance). A measure of effective information sharing was taken as the change in within-team similarity from Pre- to Mid-Discussion ratings, with higher positive values indicating that ratings across team members became more similar after some discussion; that is, participants updated their individual SA in response to information gleaned from their teammates.

Teams also collectively completed long-answer questions during their discussion session, which explored teams’ COA decisions and their consideration of risk factors. Of relevance to this paper, teams were asked to write: (Q1) a paragraph outlining the regions of Haiti that were of greatest concern and the information they used to make that determination; (Q2) a paragraph describing the risk factors they used to determine their Post-Discussion risk ratings; and (Q3) a paragraph identifying the single area of greatest concern and the risk factors underlying that decision. These questions are referred to as Q1, Q2, and Q3 in the ensuing analyses.

Cohesion was assessed using the task subscales from Carron et al.’s (1985) Group Environment Questionnaire (GEQ), with wording adapted to suit a generic (e.g., non-sport) task team. The 4-item Personal subscale assessed personal attraction to the group, whereas the 5-item Team subscale assessed group integration. All items were rated on a 9-point scale. Trust was assessed using the cognitive trust items (three items) from MacAllister’s (1995) Trust Questionnaire, rated on a 9-point scale.

Procedure

Due to COVID-19 restrictions in place at the time, all data were collected using the Opinio online survey platform (Object Planet Inc.). Team discussion sessions were conducted virtually over Zoom (Zoom Video Communications Inc.), recorded locally on the experimenter’s computer, and manually transcribed for subsequent analysis.

Prior to meeting as a team, participants individually read and completed tasks in a series of information packages, including providing Pre-Discussion risk ratings for each region of Haiti. Teams then met over Zoom to discuss the information packages and answer a series of questions as a team. Midway through the discussion session, participants received a final information package and individually completed Mid-Discussion risk ratings. They subsequently returned to the discussion session and collaboratively completed a team task, including final Post-Discussion risk ratings and the long-answer questions described above. A diagram outlining the timeline of the full experimental procedure and the relevant measures collected is provided in Figure 1.

Figure 1.

Timeline of experimental procedure and data collection.

Expert team members likewise read the full set of individual tasks in the same series of information packages, except that all experts completed the information packages and tasks for all individual roles. The expert team then met over Microsoft Teams (Microsoft Inc.) to discuss the information packages and answer a series of questions as a team, in a manner identical to the participant teams.

Results

This study combines: (1) quantitative analyses comparing participant and expert teams to explore the relationships between shared SA, team performance, effective information sharing, team cohesion, and team trust, with (2) a qualitative exploration of team discussions and written responses, to provide context to these relationships. We report the results of each approach in turn.

Participant Versus Expert Teams

Pearson correlations measured the relationships between shared SA, team performance, effective information sharing, level 1 and level 2 SA scores, team cohesion, and team trust. A diagram of significant relationships is presented in Figure 2.

Figure 2.

Diagram of significant relationships between variables of interest.

Teams that showed higher shared SA at Mid-Discussion and were also more similar to the expert team at both Mid-, r = .48, t(17) = 2.26, p = .037, and Post-Discussion timepoints, r = .46, t(17) = 2.14, p = .047, indicating positive relationships between shared SA and team performance. Higher shared SA was also positively correlated with higher average level 1 and 2 SA scores, r = .48, t(17) = 2.28, p = .036, although the direction of this relationship cannot be ascertained from the dataset; it could be that better individual SA led to the development of better shared SA, or vice versa.

Due to strong overlap in the calculations for measures of effective information sharing and shared SA at Mid-Discussion, the same relationships held true for information sharing as well, minimum r = .47, t(17) = 2.22, p = .04 for the relationship with level 1 and 2 SA scores. Effective information sharing was, moreover, positively related to both overall cohesion, r = .52, t(17) = 2.54, p = .021, and each the team, r = .51, t(17) = 2.44, p = .026, and personal, r = .48, t(17) = 2.22, p = .04, subscales, suggesting that effective information sharing supported a greater sense of cohesion among team members. The relationship between effective information sharing and team trust, however, did not reach statistical significance, r = .39, t(17) = 1.75, p = .098.

Team performance at Post-Discussion was likewise positively related to both overall cohesion, r = .58, t(17) = 2.63, p = .01, and each the team, r = .54, t(17) = 2.63, p = .018, and personal, r = .55, t(17) = 2.69, p = .016, subscales, suggesting better performance among more cohesive teams. Once again, the relationship to team trust did not reach statistical significance, r = .42, t(17) = 2.94, p = .08.

Qualitative Analyses

Team Discussions

Two of the authors (LD, HN) conducted independent preliminary content analyses of two participant team discussion transcripts to explore patterns and trends in communication among teams. The two authors then met to discuss their individual findings and draw joint conclusions. This preliminary analysis compares one team with high team performance at Post-Discussion and high cohesion ratings, against a second team with low team performance at Post-Discussion and low cohesion ratings.

The high-performing team members explicitly and purposefully provided greater opportunities for knowledge sharing, whereas the low-performing team exhibited much less structured or focused communication patterns. While one member of the low-performing team consistently attempted to engage in more structured information sharing at the beginning of the discussion, they seemed to give up once it became clear the other team members would not adhere to such information sharing behaviors. The high-performing team also demonstrated greater goal-focused discussion, whereby all team members seemed to be focused on completing the task successfully (i.e., task cohesion). By contrast, the low-performing team members did not clearly share a team goal or focus, and one team member seemed to stray away from the discussion at times and was notably less interested in gaining knowledge from their teammates.

Descriptive analyses of the communication data moreover support these findings: although the high-performing team mentioned regions of Haiti (N = 190) and specific risk factors (N = 211) more often than did the low-performing team (N = 124 and 173, respectively), their discussions were targeted toward areas of greater concern and more pressing risk factors, as evidenced by higher variability in the number of mentions (i.e., some mentioned a lot, others not at all) across all regions (SD = 15.28) and all risk factors (SD = 10.77). By contrast, the low-performing team showed lower variability in mentions for both regions (SD = 7.75) and risk factors (SD = 9.73).

Written Responses

A conceptual content analysis of all teams’ written responses to the three long-answer questions described above (i.e., Q1, Q2, and Q3) was performed. As might be anticipated from the structure of the questions, all regions of Haiti were mentioned more often in Q1 than Q3 responses. The North-East was the most frequently mentioned region in both Q1 (N = 13) and Q3 (N = 9), corroborating Chubala et al.’s (2023) finding that the North-East region bore the highest Post-Discussion risk rating averaged across teams. Other regions were less frequently mentioned in later responses, such as the South-East region, which was frequently mentioned in Q1 (N = 12) but not in Q3 (N = 2).

Likewise, frequency counts of risk factors underlying team decisions decreased from Q1 to Q3 overall. Poverty was the most mentioned factor in Q1 (N = 27) and Q2 (N = 15) but fell second (N = 11) to weather-related factors (N = 12) in Q3. This shift in focus toward weather-related risk factors likely reflects the Mid-Discussion task, wherein participants answered SA questions about the actual impacts of the hurricane’s landfall, including rainfall and wind strength.

A comparison of frequency counts in the written responses of the high-performing and low-performing teams points to important differences. The high-performing team showed a more focused consideration from the outset of the team task, with eight mentions spread across three regions of interest in Q1, compared to 11 mentions spread across six regions in the low-performing team. The high-performing team subsequently narrowed in on a single region in Q3, as instructed by the question, whereas the low-performing team seemed unable to decide between two regions.

Nevertheless, the high-performing team consistently showed broader risk consideration compared to the low-performing team, with more mentions of risk factors at each Q1 (N = 15 vs. 8), Q2 (N = 9 vs. 3), and Q3 (N = 4 vs. 3). Taken with the observations from the team discussions above, this is consistent with Chubala et al.’s (2023) finding that broader risk consideration in the Post-Discussion risk ratings was associated with greater team cohesion.

Discussion

The aim of this work was to determine factors that affected team performance in a complex C2 task where no single correct COA exists. To this end, heterogeneous teams’ responses to level 3 SA questions were compared against an expert team wherein each expert had access to all of the information from all heterogeneous roles prior to a team discussion session. Teams with better performance (i.e., greater similarity between their risk ratings and those of the expert team) also showed higher shared SA (i.e., higher within-team similarity of risk ratings after some discussion), more effective information sharing (i.e., greater within-team convergence of risk ratings after some discussion), and better team cohesion. As in Chubala et al. (2023), who found that team trust was not related to level 1 or level 2 SA, team trust was not significantly related to level 3 SA in the present study.

Because participants completed Mid-Discussion risk ratings individually, any increases in shared SA from Pre- to Mid-Discussion timepoints would be due to effective information sharing during the early part of discussion. Results showed that, to the extent that team members effectively shared their unique knowledge and updated their individual risk assessments in response, a team’s level 3 SA performance was more in line with the expert team at both Mid- and Post-Discussion timepoints. Preliminary qualitative analyses also suggested that better performance was associated with more focused and goal-oriented team discussions, a narrower focus on regions of interest in written responses, and broader risk consideration; these results add context to the quantitative findings pertaining to cohesion and information sharing.

Cohesion was related to both effective information sharing and team performance. By contrast, teams’ average level 1 and level 2 SA scores were not related to team cohesion (see Chubala et al., 2023 for more detail). The finding that level 3 SA, but not level 1 or 2 SA, relates to cohesion suggests that cohesion may play a particularly important role in the sharing and development of projection-related knowledge, which is inherently more probabilistic or subjective in nature.

The current work moreover echoes the finding presented in Chubala et al. (2023) that trust is not as strongly related to SA performance as is team (task) cohesion. One possible reason is that teams in this study worked together on a very task-focused activity, with a clear goal, for a short period of time. The clarity of the task along with the lack of time to observe the behaviors of other team members and develop interpersonal relationships may have meant there was limited opportunity for the development of trust. Furthermore, although the scenario was designed such that participants had to consider population-level risks in a fictional emergency scenario, the risk to participants themselves was minimal; given the importance of risk in the development of trust (Rousseau et al., 1998), the absence of actual risk may impede the development of trust using this scenario. Future work examining trust in more established teams operating under contexts with real risk may reveal a heightened importance for trust in the development of SA and team performance (Fors Brandebo et al., 2013).

Limitations

This study examined performance only in distributed virtual teams and may thus not necessarily reflect the development of team processes in co-located teams. However, we do not view this as a significant limitation, as many modern teams in C2 and other domains operate within distributed virtual work environments. This study also focused on ad-hoc teams; although potentially a limitation, the ad hoc nature of these teams nevertheless reflects the reality of many contemporary work scenarios where teams are often assembled on a temporary basis to address specific tasks or projects.

Finally, the design of the scenario left us to rely upon subjective, rather than objective, assessments of level 3 SA and team performance. Although this necessarily complicates the interpretation of the results, the absence of a single correct COA is a key feature of many real-world C2 missions. In these instances where objective answers may not be available, effective information sharing becomes crucial for team SA. Future work may wish to employ independent participants as expert team members, to avoid potential biases from the authors’ involvement in the development of the scenario materials and analyses of results.

Application

Taken together with the results reported in Chubala et al. (2023), this study shows that effective information sharing and team cohesion are important for the ongoing development of all levels of SA in teams. This work has implications for emergency management, military, or similar complex team-based C2 operations. For effective team functioning, considerations should be made to facilitate information sharing and promote team cohesion. We believe this to be especially critical during the planning phase of C2 missions.

We recommend that C2 teams promote cohesion by sharing, reiterating, and committing to the team’s goals and vision. In military settings, this involves providing a clear understanding of the commander’s intent and how the team’s goals support that vision. Similarly, in emergency management, clear expectations and a defined successful mission end-state will enhance cohesion. Additionally, creating opportunities for all team members to contribute, encouraging shared accountability for the team’s performance, and improving role clarity (i.e., understanding one’s and others’ roles) and role acceptance (Shoenfelt, 2011) are crucial.

These factors are evident in our preliminary comparisons of transcripts from high- and low-performing teams. High-performing, high-cohesion teams focused on the task goal and were more formalized in their information-sharing approach than were low-performing, low-cohesion teams.

Information sharing can be facilitated by team decision aid tools that compile and distill information for easy dissemination and improved understanding of the situation. Previous research on military mission planning by Hunter and Randall (2024) has shown that tools to assist mission analysis and COA development promote collaboration and shared situational awareness among team members. Such information-sharing tools are particularly beneficial for geographically distributed project team members.

Conclusion

This study highlights the importance of shared SA and effective information sharing within ad hoc emergency management teams. The findings reveal that teams with higher cohesion and better SA updating show better team performance, emphasizing the importance of these dynamics in complex C2 scenarios. By promoting team cohesion and facilitating robust information sharing, teams can enhance their decision-making processes and overall effectiveness. Future research should continue to explore these factors to develop strategies that further optimize team performance in emergency and other high-stakes environments.