Abstract
Medical teams operating under stressful conditions are prone to experiencing disconnects, which are instances of task-specific incompatibility (i.e., ineffective coordination) between team members. In simulation-based medical team training, we examined (1) whether we can identify disconnects, by locating transitions in low-level team coordination based on data collected with wearable devices, and (2) how medical teams address disconnects based on video-based annotations of different types of high-level team coordination behavior. We find that with our exploratory approach we can identify disconnects with low-level transitions. Moreover, teams resolve disconnects with explicit action and information coordination, but lose valuable time in doing so.
Introduction
Effective teamwork plays a vital role in many domains, especially in health care (Salas et al., 2007; Schmutz et al., 2019) where teams operate in complex and stressful environments, handle continuously changing workloads, and make high-stakes decisions under time pressure (Orasanu & Connolly, 1993). To optimize team performance, medical teams have to maintain effective team coordination (Helmreich & Schaefer, 1994). Team coordination can be described as an emergent phenomenon that entails the organization of interdependent actions to achieve shared objectives (Malone & Crowston, 1994; Marks et al., 2001; Salas, Sims, & Burke, 2005). Medical teams operating under stressful conditions are prone to experience episodes of ineffective coordination (e.g., Bergs et al., 2005). When teams operate in such environments, stress or arousal can produce tunnel vision. Teams may then change their view of goal achievement from a teamwork-based objective to an individual one. In turn, this may result in team members dismissing social cues, which hampers team functioning and performance (J. E. Driskell et al., 1999; T. Driskell et al., 2018; Karau & Kelly, 2004; Stachowski et al., 2009). Thus, while it is important for teams operating in stressful contexts to coordinate effectively, such teams are susceptible to ineffective coordination. Providing meaningful in situ support to these teams regarding their coordination could play a crucial part in responding to and mitigating negative consequences, for example with regards to preserving patient safety by reducing medical errors (Wiltshire et al., 2024).
Episodes during which teams fail to maintain team coordination are denoted as team coordination breakdowns. Breakdowns in coordination can lead to temporary suboptimal performance, which can ultimately interfere with a team’s goal accomplishment (Bearman et al., 2015). Such disruptions have been found to negatively affect team functioning and performance across many different domains, such as in emergency management (Grunwald & Bearman, 2017), aviation (Suzuki et al., 2008), and health care (Ren et al., 2008). Breakdowns can be considered from a more fine-grained level of detail by examining disconnects, described as distinct occurrences of incompatibilities between team members (Bearman et al., 2010). More specifically, disconnects can be identified as those instances where verbal expressions indicate that team members are cognitively misaligned (i.e. instances of incompatibility). To this end, disconnects encompass a wider range of incompatibilities than, for example, task conflicts by including rejections, contradictions, misalignments, or misunderstandings in addition to disagreements (Lehmann-Willenbrock & Chiu, 2018).
Team incompatibilities often stem from failures in team cognitive processes (Mohammed et al., 2021). These processes involve how teams collectively encode, store, retrieve, and share information. When team members develop incompatible cognitive representations (i.e., differing mental models, or knowledge structures), these processes can falter. This can lead to reduced situation awareness (Burke et al., 2006), lower transactive memory (Wegner, 1987), outdated shared mental models (Cannon-Bowers et al., 1993), or representational gaps between individual definitions of team problems (Cronin & Weingart, 2007). Relatedly, disconnects encompass both the symptoms indicating disruptions in team cognitive processes in the form of informational (i.e., asymmetric information), evaluative (i.e., asymmetric understanding), and operational (i.e., asymmetric expectations) incompatibilities (Bearman et al., 2010) as well as the outcome of such incompatibilities through verbalization of suboptimal cognitive representations.
Following complex systems theory, breakdowns, disconnects, as well as team coordination can be assessed at multiple levels of analysis. Whereas (neuro)physiological activity can be considered low level as it operates at spatially and organizationally smaller (and therefore lower) scales, behavior and the accomplishment of task goals tends to operate over larger (higher) spatial scales. The relative low versus high descriptors stem from the observation that, in complex systems, levels of a system can be differentiated within a hierarchy of organization (Sales-Pardo et al., 2007). The majority of studies on coordination and disconnects have conducted a post-hoc identification of higher-level, more readily observable episodes of ineffective team coordination (e.g., Bearman et al., 2010). However, in situ identification of ineffective team coordination at this level relies on either an internal rater, which requires a team member to allocate cognitive resources to breakdown identification, or an external rater, which is not realistic in many crisis settings such as those in health care. Subsequently, to be able to computationally identify episodes of ineffective team coordination in near real time, a growing number of studies have begun to assess a lower level of interaction, such as the movements and physiology of team members.
In previous studies, a number of approaches have emerged to estimate low-level team coordination. This level of interpersonal coordination refers to the covariation of a set of signals from a given modality (e.g., movement, physiological, or neural) across different team members, as they collaborate to manage changes in their shared working environment (Demir et al., 2018; Gorman et al., 2010; Wiltshire et al., 2019). Moreover, transitions can occur in low-level coordination. Low-level transitions can be described as significant statistical changes in properties of the low-level team coordination, which can indicate a shift from one qualitatively distinct low-level coordination state to another (e.g., from temporal similarity to dissimilarity between team members’ heart rates). Importantly, certain transitions between low-level coordination states (e.g., shifts in temporal similarity) can be indicators of ineffective coordination at the higher level (i.e., observed instances of ineffective team functioning; Amazeen, 2018; Likens et al., 2014; Zahmat Doost et al., 2024). Van Eijndhoven et al. (2023) adopted such an approach assessing transitions in low-level coordination based on heart rate and phasic electrodermal activity, to computationally identify ineffective high-level coordination in four-person teams engaged in a stressful collaborative game task. Results showed that, on the one hand, based on the located low-level coordination transitions, up to 96% of all annotated breakdowns could be identified. On the other hand, these transitions did not exclusively indicate team coordination breakdowns. In addition to indicating such breakdowns, van Eijndhoven et al. (2023) suggested that low-level coordination transitions could also indicate other phenomena, such as changes between tasks (Gorman et al., 2016) or in team affect (Gordon et al., 2021). Though this computational approach seems promising for detecting breakdowns and enabling in situ feedback regarding ineffective team coordination, the generalizability of these results to different contexts, such as health care, should be further assessed.
Thus, more research is required to assess which specific team member signals and low-level coordination states are suitable for in situ identification of high-level ineffectiveness. If in situ support could be provided based on such analyses, it could play a vital role in teams’ response to, and mitigation of, negative consequences (Wiltshire et al., 2024). Concurrently, to provide targeted support in response to negative consequences, a more detailed insight is required into what these consequences entail, as well as how they can be addressed adequately.
Consequently, with the current study we aim to explore two aspects of high-level disconnects: ways in which disconnects can be identified through a computational approach, and team events that occur after disconnects (i.e., in terms of timing-related negative consequences and high-level coordination addressment). More specifically, in the context of simulation-based medical team trainings, we first aim to examine the extent to which we can computationally identify high-level disconnects by locating transitions in different types of low-level team coordination. To this end, we adopt an approach similar to van Eijndhoven et al. (2023), assessing low-level coordination based on multiple signals that can be captured in real time (i.e., with wearable devices), and measured with Multidimensional Recurrence Quantification Analysis (MdRQA; Wallot et al., 2016), capturing repetition of patterns across team member signals. Second, we aim to evaluate when and how medical teams address disconnects based on video-based annotations of different types of high-level team coordination, categorized with the Co-ACT framework (explicit/implicit interaction regarding task-related actions/information; Kolbe et al., 2013). Such an evaluation allows us to assess adequate disconnect responses, which could be incorporated in future feedback systems providing support to teams. In addition, with an investigation regarding the timing of disconnect responses, we can study the implications of disconnects disrupting effective team coordination.
The current exploratory study involves data captured during simulation-based team trainings for medical professionals, based on which we explore the following research questions (RQs) and sub questions (SQs):
In examining these questions, we contribute both theoretically and methodologically to the research on ineffective team coordination in stressful conditions, by analyzing data captured during simulation-based team trainings from medical professionals. More specifically, following previous team research calling for a more extensive examination of meaningful dynamical approaches to identify high-level ineffective team functioning (Wiltshire et al., 2020), we contribute an in-depth investigation studying which team member signals, measures of coordination, and methods of transition detection can robustly identify high-level disconnects. Relatedly, we provide theoretical contributions furthering the integration of more traditional Team and Cognitive Science approaches assessing team coordination through observational methods, with dynamical approaches to examine team coordination from the level of signal covariation across team members. Such multi-level approaches have been encouraged by team researchers (Klonek et al., 2019; Mathieu et al., 2019) to obtain a fuller understanding of team functioning over time. In addition, research on team disconnects (Bearman et al., 2015) has identified a need for the examination of disconnects in contexts with less structured team member interactions (i.e., beyond aviation and space operations). We contribute to research on team disconnects by studying high-level team coordination responses to disconnects, as well as response timing, in the context of medical simulation-based trainings. Furthermore, we assess the generalizability of our dynamical disconnect identification approach for enabling in situ support regarding ineffective team coordination to this health care context. Taken together, these insights help narrow the gap toward providing meaningful in situ support to medical teams, assisting them in their work during real-life critical situations. Methods and materials were pre-registered and are available on the Open Science Framework: https://osf.io/q236r.
Theoretical Background
High-Level Ineffective Team Coordination
As previously stated, disconnects are denoted as verbal indicators of incompatibility (i.e., cognitive misalignment) between team members (Bearman et al., 2010). Disconnects can be part of larger episodes of ineffective high-level coordination known as team coordination breakdowns (Bearman et al., 2015). A team coordination breakdown encompasses at least one disconnect but may also constitute multiple. On the other hand, the occurrence of one individual disconnect may not necessarily be considered a breakdown, depending on its consequences regarding team functioning and performance.
Three types of disconnects have been identified as underlying instances of breakdowns (Bearman et al., 2010). First, evaluative disconnects occur when team members possess the same information, but there is a difference in the evaluation of this information, leading to incompatibility. For example, when after a joint evaluation of a patient’s status, one team member concludes the running diagnosis is “septic shock” and wants to move patient care forward accordingly, but another team member expresses doubt or rejects this diagnosis. Second, informational disconnects occur when team members possess different information, resulting in incompatibility. An example of this type of disconnect would be when some team members miss an update regarding a patient’s status and disagree with the subsequent diagnosis formulated by those who did hear the update. Third, operational disconnects occur when the actions of team members differ from what was expected by other team members, or when there is a difference in plans, leading to incompatibility. For example, when a team member wants to prioritize placing an intravenous drip, and another wants to first turn a patient on their side for further examination, leading to misaligned action plans. Studying the overarching team coordination breakdowns, as well as different types of underlying disconnects, has provided team researchers with valuable insights regarding why and how they occur.
Most research has focused on the overarching breakdowns in coordination, which has been studied across different domains, such as in aviation (Suzuki et al., 2008), health care (Ren et al., 2008; Xiao et al., 1998), emergency management (Grunwald & Bearman, 2017), and nuclear power plants (Wang et al., 2020). In the context of health care, Xiao et al. (1998) found a lack of explicit and verbal communication to be an important contributing element leading to breakdowns. Especially between team members from different disciplines (e.g., anesthesiologists and surgeons), the lack of such communication can become a breakdown trigger (Ren et al., 2008). Coordination breakdowns in health care were observed in the form of contradicting plans, insufficient problem verbalization, insufficient crisis support, and insufficient task delegation (Xiao et al., 1998). Overall, these findings indicate that breakdowns are observed when there is a gap between the team coordination that is needed and what the team actually carries out. As such, the timely detection of team coordination breakdowns fulfills an important function: it provides teams with the opportunity to discover and address discrepancies between required and actual team coordination (Easterbrook, 1995).
To better understand coordination breakdowns, some studies across multiple domains have assessed the more fine-grained disconnects underlying breakdowns. In the context of aviation, the identification and assessment of disconnects provided crucial insights in conflicts between pilots and air traffic controllers occurring during a flight (Bearman et al., 2015) and the landing of an airplane (Lai et al., 2019). Disconnects were found to cause difficulties in effectively incorporating new information into teams’ shared knowledge, resulting in discrepancies that can hinder effective team coordination. To successfully resolve their disconnects, additional explicit communication was required. Moreover, in the domain of wildfire incident management, Bearman et al. (2010) found that disconnects between sub teams of incident control, planning, logistics, operations, and public information hindered effective functioning. This, in turn, led to discrepancies in crucial information across teams, differences in plans, and delayed responses, causing valuable time to be lost in wildfire incident management. Altogether, research on team coordination breakdowns and disconnects highlights their serious implications for team functioning. To enable the provision of in situ team support at these crucial moments, previous research has turned to the analysis of low-level team coordination.
Low-Level Team Coordination
Low-level team coordination concerns the covariation of a set of signals from a given modality (e.g., movement, physiological, or neural), across different team members, while they cooperate to address task and environmental shifts (Demir et al., 2018; Gorman et al., 2010; Wiltshire et al., 2019). To quantify low-level coordination, multiple signals and measures of coordination have been assessed, which have been shown to correspond to unique high-level coordination processes such as team monitoring or dissatisfaction (for an overview see e.g., Halgas et al., 2023; Kazi et al., 2021; Ramos-Villagrasa et al., 2018). This low-level coordination can exhibit various states (Wiltshire et al., 2024). For example, a state of synchrony can describe a period during which heart rates of two people move up and down similarly (i.e., temporal similarity; Palumbo et al., 2017). Conversely, a state of asynchrony can describe heart rates of two people move up and down dissimilarly (i.e., temporal dissimilarity).
Prior work found that a state of synchrony, with high temporal similarity in electrodermal activity across team members, was associated with intense periods of high-level team member monitoring (Haataja et al., 2018) and lower self-reported team dissatisfaction (Guastello et al., 2020). Moreover, transitions from one qualitatively distinct low-level coordination state to another, represented as significant statistical changes in properties of low-level coordination, have been linked to ineffective coordination at the higher level (Amazeen, 2018; Likens et al., 2014; Van Eijndhoven et al., 2023). These transitions can be detected when the values of the low-level coordination measure deviate substantially from the preceding values, which can manifest as strong peaks and drops in the low-level coordination.
Thus, in the current study, the term levels does not refer to the distinction between individual- (describing members of a team) and team-level (describing teams as a whole) variables commonly seen in research on teams (e.g., Glew, 2009; Tims et al., 2013), but to different levels of analyses that are relevant to joint action and social activities such as those featured in teamwork (Eiler et al., 2013). Given that high-level coordination refers to behavioral interactions across team members, and low-level coordination concerns covariation in physiological signals and arm movements between team members, both low- and high-level coordination can be classified as team-level variables.
High-Level Responses to Ineffective Team Coordination
Providing meaningful and timely support to teams operating in stressful situations goes beyond identification of ineffective coordination without a human rater. Another important aspect to assess is how teams cope with ineffective team coordination. Previous studies identified different team responses to breakdowns and disconnects (Bearman et al., 2015; Grunwald & Bearman, 2017; Ren et al., 2008). These responses included: delegation, supplying additional resources, mentoring, asserting authority, switching or replacing team members, and joint problem solving. To further categorize teams’ high-level responses to ineffective coordination, they can be assessed with the Framework for Observing Coordination Behavior in Acute Care Teams (Co-ACT; Kolbe et al., 2013). This framework consists of four quadrants organized along two dimensions (explicit vs. implicit coordination; action vs. information coordination), shown in Figure 1.
Overview of high-level team coordination quadrants.
As indicated in Figure 1, each quadrant encompasses a different type of high-level team coordination, specified with underlying behavioral codes. Examples of high-level explicit action coordination are communication regarding instructions or planning. Implicit action coordination can for example be observed as monitoring of other team members or providing assistance. Information requests and evaluation are examples of explicit information coordination. Examples of behavioral codes for implicit information coordination are provision of information without request and team members gathering information within their environment.
Nonetheless, previous work on breakdowns and disconnects did not extensively assess what form of high-level coordination teams eventually engage in to respond to specific types of disconnects. Moreover, given that effective team performance in high-stakes environments not only requires disconnect responses to be adequate but also swift (Kozlowski et al., 1999), it is important to assess whether there is a delay in disconnect responses and how long that delay would be. These details should be further studied in the context of health care to ensure meaningful support can be provided to mitigate the negative consequences of ineffective team coordination.
Method
The dataset used in the current paper is part of a larger research project. It involves low- and high-level team coordination data and disconnects collected during a simulation-based medical team training in two hospitals. For the purpose of the current study, we focus on data of training scenarios that included annotated disconnects between team members. More detailed information regarding the larger research project can be found in the preregistration on the Open Science Framework (https://osf.io/q236r).
Participants
For the current study, we analyzed data of 12 teams participating in simulation-based medical team training. In total, these teams consisted of 80 females and 6 males, with the youngest participants falling within the age range of 18 to 24 years old, and the oldest falling within the age range of 55 to 64 years old. Data for each team was collected during one or two scenarios, resulting in a total of 14 scenarios, with teams ranging from five to eight members. For the two teams that participated in two scenarios, one team had an overlap of four out of six members across scenarios, and the other had an overlap of one out of six members. Teams participated in trainings from either hospital A or B. Teams from hospital A were all employed in an obstetrics and gynecology department, which consisted of nurses, a resident, and a gynecologist, and some included student nurses and midwives. Teams from hospital B were either part of the pediatrics or emergency room department, and were composed of nurses, pediatricians, and pediatrician assistants.
Quasi-Experimental Procedure
We collected data during simulation-based medical team trainings, which the hospitals were already providing. We informed participants of the training about our study 3 to 7 days in advance, with an information letter distributed by their trainer. At the beginning of the training, we asked participants if they had questions regarding the research, and if agreed upon, to sign a consent form. Then, we provided participants with Empatica E4 wearable devices (Empatica Srl, Milan, Italy) that collect PPG (reflecting blood volume change arising from heart beats; 64 Hz), EDA (reflecting electrical conductance properties of the skin; 4 Hz), and arm movement (reflecting acceleration across three axes; 32 Hz). Subsequently, participants completed a demographic questionnaire. To collect 360° video and audio data, we utilized a GoPro Max 360 camera (GoPro, Inc., California, United States). After starting the camera recording, the training would continue as usual, without the immediate presence or involvement of the researchers.
During each scenario, a medical case was simulated. All scenarios involved an unforeseen medical emergency that teams had to respond to. In hospital A, actors (student doctors), a hybrid birthing simulator, and a neonatal mannequin were utilized in the scenarios. The medical case would either involve the delivery of a baby in breech position (N = 6), or a pregnant woman with pneumonia that would go into septic shock (N = 5). In hospital B, a newborn or toddler mannequin was utilized as a patient. The medical case involved respiratory syncytial virus (N = 1), hypovolemic shock (N = 2), or supraventricular tachycardia (N = 1). All scenarios were analyzed from the time point when the whole team was present, until the trainer ended the scenario. Subsequently, a scenario lasted 10 min and 3 s on average (SD = 4 min 50 s). Scenarios took place in a realistic hospital setting, containing all the usual medical equipment. Team coordination was required for all scenarios to reach a positive patient outcome.
Once a simulated scenario ended, prior to the debrief, we asked participants to complete a questionnaire regarding team- and task-work (Johnson et al., 2007), team adaptive performance (Marques-Quinteiro et al., 2015), team potency (Guzzo et al., 1993), and perceived stress (Amirkhan, 2018). The hospital’s trainer completed a questionnaire regarding the teams’ performance (Frankel et al., 2007). Next, the trainer and participants did a debriefing of the training. Once all scenarios were done, we gave participants the opportunity to ask any questions regarding the study, concluding the quasi-experiment.
Annotating High-Level Team Coordination and Disconnects
Two independent annotators manually annotated audiovisual scenario recordings for high-level team coordination and disconnects. They utilized the Framework for Observing Coordination Behavior in Acute Care Teams (Co-ACT; Kolbe et al., 2013) to annotate high-level team coordination. This framework consisted of four quadrants (explicit action, explicit information, implicit action, implicit information). Each quadrant included subsequent behavioral codes with examples from teams operating in different domains, such as aviation, surgical, or military teams. We modified examples to better suit the context of the training scenarios (see https://osf.io/5rdtk for a detailed description and the codebook). Given our current research questions, we focus on quadrant-level annotations of team coordination. Finally, we annotated the absence of high-level team coordination when no such behavior was present.
To annotate disconnects based on our audiovisual scenario recordings, we created a codebook based on the descriptions of evaluative, informational, and operational disconnects, identified by Bearman et al. (2010). We adjusted codebook examples to suit the context of the current study. With regard to information-based incompatibilities, we annotated two types of disconnects. Evaluative disconnects represent distinct instances of incompatibility as a result of a difference in information appraisal. On the other hand, informational disconnects entail instances of incompatibility caused by team members possessing different information. Action-related incompatibilities were identified as operational disconnects. Such disconnects arose from differences in expectations regarding actions or action planning. The start of a disconnect was operationalized as the beginning of the sentence that initiates the incompatibility (e.g., “I want a second intravenous drip, with a complete lab report.”). Conversely, the end of a disconnect was identified as the end of the last sentence of the incompatible other team member (e.g., “But she is instable due to low tension, so what I think we first need to do is increase tension.”). In addition, disconnect resolution was considered to occur from the moment teams engaged in high-level team coordination that addressed the corresponding incompatibility. The audiovisual recordings were revisited to examine whether annotated instances of high-level team coordination took place to address disconnects. A detailed description of the disconnects and examples can be found here: https://osf.io/5rdtk.
Two independent annotators assessed the presence of each high-level team coordination and disconnect type at a per second rate. This resulted in two annotation timeseries at a resolution of 1 Hz per scenario. Annotators had a background in Biomedical Science, or Cognitive Science and Artificial Intelligence. To start their training, the annotators extensively reviewed the codebook together with the researcher that created it and a trainer of the simulation-based team trainings, after which the annotation process started. Annotators were able to repeatedly review the audiovisual recordings for their annotation assessment. Annotation differences and ambiguity in the codebook were evaluated after annotating ~5%, ~10% and ~20% of all scenarios. Next, inter-annotator agreement was computed with Cohen’s Kappa (κ), which indicated substantial agreement. Consequently, the subset of annotations was expanded to a ~50% subset (N = 12), and agreement was recalculated (see Table 1).
Overview of Inter-Annotator Agreement Measured as Cohen’s Kappa (κ).
To resolve instances of code disagreement between annotators, a final decision was made by the primary researcher that created the codebook and trained the annotators. Finally, we generated one unified high-level team coordination timeseries (sampled at 1 Hz) for each scenario.
Identifying Transitions in Low-Level Team Coordination
To obtain low-level team coordination based on three different signals, we recorded and preprocessed arm movement, PPG, and EDA signals. These signals can easily be captured in real time with wearable devices, which is an important characteristic to enable in situ support based on low-level team coordination. Moreover, transitions in low-level coordination based on EDA and PPG have been found to indicate episodes of ineffective team functioning (Van Eijndhoven et al., 2023). Transitions in low-level coordination based on arm movement could correspond to changes in the alignment of arm movements. Such alignment can facilitate mutual understanding in interaction (Rasenberg et al., 2020). To each of these signals, we applied sliding window MdRQA (Wallot et al., 2016). Transitions in the resulting low-level team coordination data were assessed with a sliding window nonlinear prediction algorithm (NLPA; Gorman et al., 2020; Sugihara & May, 1990).
Preprocessing
The accelerometer of the wrist worn E4 wearable collected arm movement. Movement data was captured along three axes X, Y, and Z, each representing the axis’ magnitude and direction of acceleration. We conducted preprocessing following Lehmann-Willenbrock and Hung (2023), resulting in a unified measure of the amount of overall arm movement (i.e., magnitude at 4 Hz) for each team member, in each scenario. A more detailed description of the implemented preprocessing methods, including software libraries, can be found in Appendix A.
PPG signals were preprocessed following Elgendi et al.’s (2013) and van Gent et al.’s (2019) methods. For the extraction of heart rate (beats per minute) from these PPG signals, a sliding window technique was used. This process involved a window size that included PPG data equivalent to 120 s, which was slid over the teams’ PPG signals. Heart rates were calculated based on data within these 120 s windows, after which the window was slid forward by 1 s, and heart rate was recalculated. For each team member in each scenario, we acquired a continuous time series of heart rate values (1 Hz).
The preprocessing for the raw EDA signal was based on Westerink et al.’s (2020) work. The phasic component of the EDA signal was extracted, as it has been found to represent event-related sympathetic activity (Benedek & Kaernbach, 2010), and has been utilized in previous team coordination research (e.g., Ahonen et al., 2018; van Eijndhoven et al., 2023). Finally, we obtained a phasic data timeseries (4 Hz) for each team member in each scenario.
Measure of Coordination
After pre-processing the signal data, we applied sliding window Multidimensional Recurrence Quantification Analysis (MdRQA; see Wallot et al., 2016) to generate a continuous measure of low-level team coordination. This method can be utilized to extract coordination patterns of multiple variables over time, by embedding the data in a multidimensional phase space. For example, per scenario, a window with a size of 60 s and step of 1 s was slid over the EDA signal timeseries of each team member, and any EDA signal data falling within each window was mapped into a multidimensional phase space. By locating recurrent points (i.e., points at which all mapped signals occur within a given radius), as well as repeating sequences of adjacent points within the phase space, a recurrence plot can be generated. Within such plots, black squares are markers of recurrent points. Figure 2 shows an example of the sliding window method (a) for the application of MdRQA and a subsequent recurrence plot (b).
Overview of example analysis involving two-member team’s heart rates and MdRQA.
Based on this recurrence plot, different metrics of team’s coordination dynamics can be computed, such as the percentage of recurrent points or the average length of recurrent sequences (for a complete overview of metrics see Wallot and Leonardi, 2018). Following van Eijndhoven et al. (2023) and Gorman et al. (2020), we focused on the Determinism (DET) metric, which represents the amount of recurrent points that occur in sequences divided by the total number of recurrent points. In other words, DET provides information regarding the predictability of a system. It measures how often patterns in the data repeat in a regular, structured way, instead of at random. Figure 2c shows an example of a DET timeseries following the sliding window MdRQA briefly detailed above. The DET metric ranges from 0 to 1. Values approaching 0 reflect irregularity in the signals (i.e., few repeating patterns), and values approaching 1 reflect regularity (i.e., many repeating patterns). For example, if 10 recurrent points are located, out of which 8 points occur in a diagonal sequence with other recurrent points, the DET would be 8/10 = 0.8. This DET value indicates that a majority of recurrent points belong to longer sequences of recurrent values, suggesting regularity in signal patterns over time. This was also the case for the recurrence plot shown in Figure 2b, based on which a DET value of 0.88 was calculated (displayed at the 60 s mark in Figure 2c). We generated one DET timeseries sampled at 1 Hz for each signal type per scenario. Figure 3 provides an overview of the low-level team coordination types. Details regarding the software libraries utilized to run MdRQA as well as our parameter settings can be found in Appendix B.
Overview of low-level team coordination types.
Transition Identification
We further analyzed low-level team coordination data to locate transitions. To this end, we applied a sliding window nonlinear prediction algorithm (NLPA; Sugihara & May, 1990), which has previously been utilized to identify transitions in low-level coordination (Gorman et al., 2020; Van Eijndhoven et al., 2023). This method splits any provided window of data (e.g., of DET-based low-level coordination data) in two subsets (Sugihara & May, 1990). A nonlinear model is created based on the data points in a first subset of DET data, taking into account trajectories of neighboring data points, to make predictions for each DET data point in the second subset. The model predictions are evaluated based on calculations of Root Mean Square Error (RMSE), representing the average distance between predicted and actual values. Windows of data with RMSE values more than two standard deviations from the mean were identified as transitions (similar to Gorman et al., 2020). Consequently, the identified transitions reflect significant increases or decreases in regularity of the DET low-level coordination data. Appendix C describes the software libraries and parameters settings that were implemented to run the NLPA.
Comparing Low-Level Transitions to High-Level Disconnects
To examine the extent to which low-level transitions correspond to high-level disconnects, we calculated recall, precision, and F1-scores. Given that low-level coordination transitions were calculated per scenario (i.e., for each scenario we calculated new parameter values for the measures of coordination and methods of transition detection), we first obtained these result scores per scenario. Afterwards, we averaged recall, precision and F1-scores to reflect the averages across scenarios. Recall scores provided insight into the percentage of disconnects that could be identified by locating low-level transitions. Precision was informative regarding what percentage of low-level transitions coincided with a high-level disconnect. F1-scores represent the balance between recall and precision.
In order to calculate these scores, we needed to obtain true positives, false negatives, and false positives. To determine true positives, we followed van Eijndhoven et al.’s (2023) approach. Low-level transitions, that occurred 10 s before, during, or 10 s after a high-level disconnects, were marked as true positive. Figure 2c, at the 154 s mark, shows an example of a true positive. These time windows before and after the actual disconnects occurred, were included as transitions could precede (Dias et al., 2019) and succeed (Gorman et al., 2020) ineffective team coordination. We subtracted the number of true positives from the number of located low-level transitions to obtain the number of false positives. An example of a false positive can be seen in Figure 2c, at the 118 s mark. We obtained the number of false negatives by subtracting the number of true positives from the number of high-level disconnects. Finally, recall was calculated as
Overview of the Different High- and Low-Level Constructs Involved in the Current Study.
Assessing High-Level Team Coordination Patterns of Disconnect Resolution
A three-pronged approach was set up, to study the patterns of high-level team coordination that teams tended to engage in for disconnect resolution. We first assessed whether a type of high-level team coordination tended to occur immediately after disconnects as compared to no high-level team coordination occurring immediately after (SQ2.1). To this end, we extracted the annotation sequence from our annotation timeseries, as illustrated in Figure 4.
Illustration of annotation sequence extraction based on an annotation timeseries example.
Figure 4a shows the abbreviations utilized in 4b and 4c. Figure 4b shows an annotation timeseries example, involving multiple sub teams that were formed during a scenario (e.g., nurses taking care of medication while gynecologist and obstetrician examine patient). Low-level coordination was analyzed using signal data derived from the full team. In contrast, annotations of high-level coordination were based on observed coordination behavior across the full team or within a subteam. Figure 4c provides a visual example of how an annotation sequence was extracted from an annotation timeseries. Given that we were only interested in disconnect-related annotations, unrelated annotations were excluded from the sequence, also illustrated by dashed abbreviations in Figure 4b (top row), omitted in Figure 4c.
Subsequently, we applied Lag Sequential Analysis to the annotation sequences involving annotations based on both the whole team and sub teams, to assess the temporal association between disconnects and annotations of high-level team coordination. We used the function seqgroups() of R library LagSequential (Draper & O’Connor, 2019), with the lag parameter set to 1. This function was specifically designed to compute sequential analysis statistics for data consisting of multiple segments, and in our case, multiple teams in different scenarios. Second, in case episodes of no annotated high-level team coordination tended to occur immediately after disconnects, we wanted to identify what type of high-level team coordination teams first tended to engage in, to resolve disconnects by addressing the team member incompatibilities (SQ2.2). Consequently, we excluded periods of no annotations after disconnects from our annotations sequences, and reran Lag Sequential Analysis. Third, we extracted the timing of disconnect resolution, which indicated the time it took teams to engage in high-level team coordination to address disconnects (SQ2.3).
Results
We identified disconnects in high-level team coordination and transitions in low-level team coordination to further assess their correspondence. An overview of descriptive information regarding the resulting transitions and disconnects can be found in Table 3.
Overview of Identified High-Level Disconnects and Low-Level Team Coordination Transitions.
Note. Means and SDs were calculated at the level of scenarios. For 2 out of 12 teams, data was collected during 2 scenarios, resulting in the collection of data during 14 scenarios in total.
Table 3 shows that across low-level team coordination types, twice as many transitions were identified compared to annotated disconnects. In addition, the duration of transitions identified in any type of low-level team coordination was around half the length of annotated disconnects.
Recall, precision, and F1-scores were computed to examine the extent of correspondence between low-level transitions and high-level disconnects (RQ1). The results for each low-level team coordination type are visualized in Figure 5.
Results per type of low-level team coordination including 95% confidence intervals.
As illustrated in Figure 5, transitions in low-level coordination DET based on arm movement yielded the highest recall, precision, and F1-score as compared to the other low-level coordination types, though the differences were small. In addition, we observed a similar pattern of results across types of low-level team coordination. Recall was found to be highest, ranging from 0.75 to 0.86. This indicates that up to 86% of annotated high-level disconnects coincided with a transition in low-level team coordination. F1-scores, which reflect the balance between recall and precision, and thus appeared in-between those scores, were found to range from 0.5 to 0.59. Among the result metrics, precision scores were found to be lowest, ranging from 0.39 to 0.46. Our precision scores show that up to 46% of identified transitions in low-level team coordination coincided with annotations of high-level disconnects. These results indicate that low-level transitions do correspond to high-level disconnects, but not exclusively.
Consequently, we reassessed moments in the audiovisual recordings, during which transitions in low-level team coordination occurred that did not indicate high-level disconnects. Figure 6 shows a categorization of events observed during these transitions, and their frequency.
Overview of observed events cooccurring with transitions in low-level team coordination.
As Figure 6 shows, baby-related events were observed at moments during which low-level transitions occurred. Baby-related events entailed all events during which the teams’ main focus was on the baby. For example, when the baby mannequin’s head started to crown, or when the medical team learned that the baby was in breech position. Patient-related events included all events during which the teams’ main focus was on the patient. Examples are, when the birthing mother lost hope and needed to be encouraged to keep pushing, or when the medical team learned that the patient’s saturation was decreasing rapidly. Information evaluation includes instances of explicit communication during which the current status of the situation (e.g., the patient or related tasks) were assessed. This category coincided most frequently with transitions in low-level team coordination. The category subtasks to central or vice versa includes moments during which previously formed sub teams merged back into a unified team. An example is when a team is broken up into sub teams taking care of anesthetics, the birthing mother, and the baby, but then comes together to centrally discuss the status of the mother. In addition, this category includes moments during which a unified team breaks up into smaller sub teams. A unified team can for example be engaged in a central evaluation of next steps and then break up into sub-teams to perform the discussed steps. In few cases, disruptions external to the team (e.g., team member receives training unrelated phone call), or switches from baby birthing to newborn baby care coincided with low-level transitions.
During each of these events, at least one team member experienced a shift from one task or task phase to another. Very straightforward examples of task shifts include switches from individual to central tasks and vice versa, and switches from birthing to newborn care. All external disruptions and baby- and patient-related events involved incidents that required a swift response, and thus a change in tasks, from at least one team member. Examples of these respective event categories are, a patient’s guest interrupting a team discussion with questions, a baby having issues breathing, or a patient moving into a painful contraction. Finally, information evaluations were categorized as periods during which teams assessed their current status and made plans for upcoming taskwork. Such periods can be considered transition periods, which encapsulate shifts from one task or task phase to another (Bush et al., 2018; Marks et al., 2001).
Thus, following RQ1, we find that up to 86% of high-level disconnects can be identified based on transitions in low-level coordination. In addition, up to 46% of all low-level transitions point to disconnects, while the other 54% are indicators of task shifts, mostly relating to information evaluation.
Next, we assessed if there was a greater than chance probability of a disconnect immediately being followed by some form of high-level coordination instead of no high-level coordination utilizing Lag Sequential Analysis (SQ2.1). In addition to Lag Sequential Analysis on the full dataset of 14 scenarios, we ran four additional analyses with a subset of 12 scenarios, systematically leaving out one scenario from both teams such that all participants across scenarios were unique. In terms of statistical significance, results based on the subsets consisting of 12 scenarios were the same as those of the full dataset consisting of 14 scenarios. Consequently, we report results based on the full dataset.
Table 4 shows an overview of the annotations that were observed following a disconnect. Z-scores were assessed to determine which annotation sequences occurred at a greater than chance rate (Bakeman & Quera, 2011). Sequences resulting in z-scores above 1.96 or below −1.96 were considered significant. Our results indicate that after disconnects, teams were significantly more likely to experience a period in which no high-level team coordination was annotated (z = 4.87, p < .01). In other words, the majority of disconnect occurrences was not followed immediately by any high-level team coordination.
Lag Sequential Analysis Results for Disconnects Followed by Either a Type of High-Level Team Coordination, or an Episode of No Annotated Coordination at Lag 1.
Note. Bold font indicates statistical significance.
To get a better idea of the first type of high-level team coordination that teams tend to engage in after disconnects (SQ2.2), we removed periods of no annotations immediately following disconnects from our annotation sequences. We reran the Lag Sequential Analysis for disconnects in general and for each type of disconnect separately. Results are shown in Table 5.
Lag Sequential Analysis Results for (General and Type-Specific) Disconnects Followed by the First Occurring Type of High-Level Team Coordination at Lag 1.
Note. Bold font indicates statistical significance.
Table 5 shows that following disconnects, teams more frequently engage in explicit than implicit coordination. After the occurrence of disconnects in general, teams were significantly more likely to first engage in explicit information (z = 4.86, p < .01) and explicit action (z = 4.51, p < .01) coordination. More specifically, teams were significantly more likely to follow up on evaluative and informational disconnects with explicit information coordination (evaluative; z = 4.08, p < .01, informational; z = 7.19, p < .01), and operational disconnects with explicit action coordination (z = 6.60, p < .01). To sum up our findings regarding SQ2.2, after disconnects teams are more likely to engage in explicit than implicit coordination, with evaluative and informational disconnects typically being followed by explicit informational coordination, and operational disconnects by explicit action coordination.
Finally, through an examination of the audiovisual recordings, we extracted the time it took teams to engage in high-level team coordination to address disconnects (SQ2.3). Figure 7 visualizes our findings.
Boxplot of response times to engage in high-level team coordination after disconnects.
As Figure 7 indicates, the median response times across disconnect types was found to be between 2 and 3 s, and the majority of response times to be between 0 and 7.25 s. A small number of outliers were detected for response times of informational and operational disconnects, with the largest outlier representing a response time of 22 s. These results suggest that for the majority of disconnects, medical teams showed a delayed response to addressing disconnects, lasting multiple seconds. This time window may be critical in contexts where every second counts, such as in medical emergencies.
Discussion
In the current exploratory study, we gained multiple insights regarding high-level disconnects within the context of simulation-based medical team training, which contribute important details toward achieving adequate in situ team support. Focusing on the in situ aspect of providing team support, we assessed approaches for the computational identification of disconnects. To gain a deeper insight into what support may be helpful, we examined how teams address these instances of ineffective coordination in terms of timing and high-level team coordination. First, we found that we could identify the majority of high-level disconnects based on low-level transitions (RQ1), adopting van Eijndhoven et al.’s (2023) approach. These transitions were not necessarily related to high-level disconnects but did correspond with events during which at least one team member experienced a shift from one task or task phase to another. Second, evaluating how medical teams addressed disconnects (RQ2), we found that a significant amount of disconnects was not instantly succeeded by any type of high-level team coordination (SQ2.1). These delays in response to disconnects were found to last from 1 up to 22 s, with a majority of delays lasting around 2 to 3 s (SQ2.3). In addition, our results also showed that overall, teams responded to disconnects with explicit coordination more frequently than implicit coordination (SQ2.2).
Our findings indicate that disconnects tend to co-occur with transitions in low-level team coordination, which suggests that locating these transitions can be a promising approach for the identification of ineffective high-level team coordination. Developing and examining such an approach is an important and necessary first step toward using continuous measures of low-level team coordination, and transitions therein, to monitor and support high-level team processes and states. However, low-level transitions were also found to indicate other team events, particularly events relating to functional task- or task-phase shifts involving at least one team member. Previous literature suggests that such task shifts can disrupt high-level team coordination (Smith-Jentsch et al., 2015) and can also be related to changes in low-level team coordination (based on neural signals and communication; Gorman et al., 2016). Similarly, our results showed that low-level coordination transitions (based on physiological signals and arm movement) can relate to high-level disconnects, as well as task shifts. For example, on the one hand, low-level coordination captured with arm movements can correspond with motion alignment, which occurs when team members mimic or repeat other member’s arm movements. This kind of alignment may facilitate mutual understanding (Rasenberg et al., 2020). Changes in alignment may in turn capture fluctuating demand for increased mutual understanding. These fluctuations in demand can occur during disconnects, which encompass a wider range of team member incompatibilities, including disagreements, misalignments, and misunderstandings, requiring increased mutual understanding to be resolved. On the other hand, arm-movement based coordination can also correspond with task-related changes requiring arm movement. These findings offer a threefold contribution to theoretical implications regarding multi-level team coordination. First, they suggest that low-level transitions, at least to some degree, may relate to both functional (i.e., in response to task shifts) and dysfunctional (i.e., disconnects) team functioning. Second, our findings corroborate van Eijndhoven et al.’s (2023) results, in that we also found a pattern of higher recall and lower precision across types of low-level team coordination, similar to their results, indicating generalizability of the approach to different contexts (i.e., a collaborative game task vs. simulation-based medical team training). Third, they provide a deeper insight into which team member signals, measures of coordination, and methods of transition detection are suitable for the identification of high-level disconnects. Hence, our findings in relation to RQ1 contribute practical implications toward the development of a more generally applicable system to monitor and support high-level team coordination.
Moreover, our findings contribute valuable theoretical implications with regards to high-level team coordination responses to disconnects and response timing, in the context of medical simulation-based trainings. We found that disconnects are addressed with explicit coordination, which aligns with previous findings that explicit communication plays a significant role in medical team performance in general (Kolbe et al., 2012; Kolbe et al., 2014) and in dealing with unexpected events (i.e. medical emergencies) in particular (Rico et al., 2019; Rico et al., 2020). In addition, our findings are in line with previous studies, which suggest that explicit communication plays a pivotal role in the resolution of both team coordination breakdowns (Xiao et al., 1998) and disconnects (Lai et al., 2019). The current study adds additional insight by showing that the nature of disconnects influence the type of response teams engaged in. More specifically, after evaluative and informational disconnects teams tended to engage in explicit information coordination, and after operational disconnects they tended to engage in explicit action coordination. Given that information-related incompatibilities form the basis for evaluative and informational disconnects, resolution by explicit communication aimed at increasing mutual understanding makes a logical next step (i.e., explicit information coordination).
Similarly, it makes sense for operational disconnects, which are based on action-related incompatibilities, to be resolved by explicit communication aimed at organizing joint actions (i.e., explicit action coordination). Besides observing that our results align with those of prior research, we also uncovered divergent results compared to previous studies. Unlike Bearman et al. (2015), who found that not all disconnects were resolved by the teams, we observed that teams resolved all disconnects. Whether and how disconnects get resolved might be dependent on the type of teams in which they occur. In the case of Bearman et al.’s study (2015), disconnects were observed within incident management teams consisting of a mix of professionals and volunteers, coordinating emergency response during wildfires. In such teams, levels of training may not be as uniform as they are in professional medical teams, making it more difficult to resolve disconnects. Thus, with our findings related to SQ2.2, we offer practical implications by providing a more fine-grained insight into the resolution of different types of disconnects, which can be incorporated in the form and function of support provided to teams.
Furthermore, while time is of the essence in time-critical contexts such as medical emergencies, few studies have examined response times to disconnects. The current research shows that, in the majority of cases, the medical teams lost multiple seconds before starting to address their disconnects. This was especially the case for evaluative disconnects, for which response times were found to be longest. In part, these delays could be related to team members’ reaction time to previous disconnects. Extant research showed that individual response times following perturbations during cognitively demanding tasks can vary from 1.12 to 1.43 s (Inkol et al., 2018). Though a majority of disconnects were resolved relatively quickly (within 2 to 3 s), they did last longer than these cognitive response times found by Inkol et al. (2018), with some disconnects even taking 10 to 20 s longer to be resolved. However, these cognitive response times were measured during individual non-medical tasks, whereas the delays in responding to disconnects we identified were obtained in a team medical setting, which may affect cognitive response times. Additionally, perturbations in our quasi-experiment were of cognitive nature (i.e., disconnects), while perturbations in Inkol et al.’s (2018) experiment were physical. To increase our understanding of how response times may be affected, future research should measure and examine response times in different settings (i.e., individual, dyadic, teams of different sizes, varying task contexts) and in relation to perturbations of different kinds. Nonetheless, our results following SQ2.1 and SQ2.3 offer practical implications that corroborate the potential value of a system that enables in situ identification of disconnects and provision of quick support to teams. Such a system could in turn facilitate a quicker response.
Study Limitations
There are several limitations to our research. First, our data was collected during a simulation-based medical team training. Although these involve professional staff, circumstances surrounding those trainings might not entirely be the same as during a real-life crisis situation. However, data collection during such real-life crises induces multiple complications, including issues of patient privacy. Consequently, data was collected during a highly realistic simulation of real-life stressful medical situations, which has several features in common with actual real-life medical situations, contributing to the ecological validity. For example, participants of the trainings were part of the medical team in their actual professional roles, wearing their normal work attire. These trainings took place in a realistic hospital setting, containing all the usual medical equipment. All training scenarios included an unforeseen medical emergency, requiring a timely and adequate response. Similar to crisis response in real life, team coordination was required to achieve a positive patient outcome. Moreover, previous work has argued for the ecological validity (Tannenbaum & Yukl, 1992) of carefully designed simulation-based trainings (Salas, Wilson, et al., 2005), which can improve team functioning, and in turn, patient safety. Such trainings offer teams in healthcare a safe environment to practice team coordination in response to unforeseen medical emergencies (Salas et al., 2007). Consequently, we believe our quasi-experimental set up approaches a realistic crisis setting.
Second, in some cases during the trainings, multiple sub teams were formed, leading to more than one interaction taking place at the same time. On average, annotators reported that for 30 s, spread throughout a scenario, they experienced difficulties following each separate interaction. Oftentimes, such noisy periods were also paired with noise from the medical equipment or the patient. Though the annotators worked to the best of their ability, it could be the case that during these seconds, an annotation was missed. In the case of a further and similar data collections, researchers can consider wearable devices with microphones, in an attempt to more clearly capture interactions within sub teams.
Third, though data was collected throughout the whole duration of the trainings, we did not examine the full training data in our analysis. Training scenarios started with one or two team members examining the patient, after which other members were gradually called into the training room based on their expertise. Analysis of all team member signals before the whole team was part of the scenario interactions would not provide us with meaningful insights regarding overall low-level team coordination. Researchers conducting similar data collection should consider a system that keeps track of which team members enter the training room, so that analysis can only involve those taking part in the scenario.
Future Research
Following our current study, we identified multiple future research directions that were beyond our study’s scope. For example, while our study focused on the most widely assessed team member signals, from wearable sensors, in research on team coordination (Halgas et al., 2023), including other signals (e.g., communication-based signals such as speech activity; Zhou et al., 2020) might be useful for the identification of high-level disconnects. Similarly, future research should involve additional measures of coordination and transition detection methods. As each of these components within our approach contributes different characteristics to the operationalization of low-level team coordination and identifying transitions, researchers might be able to more precisely differentiate between low-level transitions related to disconnects and task shifts. In addition, future research should consider the integration of Machine Learning, involving algorithms that are particularly suitable for the detection of patterns in timeseries (e.g., Recurrent Neural Networks; Shih et al., 2019). Such algorithms could further help distinguish between functional and dysfunctional transitions.
Moreover, in our work we focused on the first step in the development of an in situ support system, which is establishing computational dynamical approaches for the in situ identification of ineffective team functioning. For future studies, we plan to conduct research regarding the next step, which encompasses how feedback can effectively be delivered to teams regarding their ineffectiveness (Wiltshire et al., 2024). In turn, this may result in a faster resolution of coordination breakdowns and disconnects. Complex systems theory and empirical work (Scheffer et al., 2012) suggest that during certain transitions in systems’ low-level coordination, they may be more receptive to guidance toward a preferred system-level behavior. Consequently, particularly for these instances of suboptimal high-level team coordination, during which transitions in low-level coordination occur, providing feedback might be effective. To start our examination into team feedback, inspiration can be drawn from literature on, for example, metacognitive prompting, which involves cues to stimulate task characteristics awareness, performance strategies, and outcomes evaluation (Wiltshire et al., 2014; Winne, 1997). Such prompting is found to improve decision making and knowledge acquisition, and relate to lower self-ratings of cognitive load (Fiorella et al., 2012; Fiorella & Vogel-Walcutt, 2011). However, many questions regarding the form and function of in situ team feedback delivery remain unanswered. For example, is in situ feedback most effective when delivered to individual team members, dyads, or the whole team? Moreover, through which modality of delivery is in situ feedback most effective? Such knowledge would be vital for a meaningful in situ support system for teams.
Though the current study examined the negative implications of verbal expressions of incompatibility (including disagreements, contradictions, misalignments, misunderstandings) among team members, such incompatibilities could also have positive implications. Lehmann-Willenbrock and Chiu (2018) identified multiple benefits of team member incompatibility. For instance, incompatibilities can lead to an increase in attention and idea sharing of team members (De Dreu & West, 2001; Nemeth & Rogers, 1996). Furthermore, the occurrence of more incompatibilities that are considered part of the regular problem-solving process without severe consequences (Angouri, 2012) increases the likeliness of successful problem solving (Chiu & Khoo, 2005). In addition, incompatibilities create an opportunity for team members to learn from each other (Van Offenbeek, 2001). The examination of such implications was outside the scope of the current study, as we focused on response delays after disconnects. Such delays were considered a negative implication of disconnects, as in time-critical contexts an unlikely but ideal scenario would be for team functioning to always be optimal. In this case, the occurrence of any disconnect would steer teams away from the ideal scenario, and subsequent response delays would be considered a negative implication. Given the lack of studies regarding this implication of disconnects, and the importance of timely responses to ineffective team functioning in time-critical contexts, such a study was important to conduct. To gain a more complete understanding of disconnect implications, future studies could more closely examine periods of team functioning after disconnects beyond the scope of response delays. Such studies could take into account a wider range of implications identified by previous research, including examination of attention increases or idea sharing.
Furthermore, an analysis of the gathered questionnaire data on team- and task-work (Johnson et al., 2007), team adaptive performance (Marques-Quinteiro et al., 2015), team potency (Guzzo et al., 1993), and perceived stress (Amirkhan, 2018) from participants, and the trainer’s teams performance ratings (Frankel et al., 2007), was also outside of the scope of the current study. While the inclusion of self-reported data could provide interesting findings regarding team coordination, we focused on extensively examining dynamical measures of low-level team coordination in the current study. We plan to incorporate the questionnaire data in future analyses that could, for example, explore the relationship between trainer ratings of team performance and disconnect responses in terms of high-level team coordination, as well as response timing. Other examples include an analysis of disconnect response timing in relation to self-reported team potency or team adaptive performance, or of perceived stress and the amount of annotated disconnects.
To gain a more complete understanding of transitions in low-level team coordination, future research should work toward building a taxonomy that not only considers high-level team coordination, but also factors related to changes in task properties. This taxonomy should consider transitions in an elaborate range of low-level team coordination types (i.e., based on a variety of signals and measures of coordination). Conversely, such an overview could provide inspiration for innovative ways of more precisely identifying disconnects, which is a necessity for enabling robust in situ feedback regarding ineffective team coordination. For example, it can provide insight in what signals more exclusively relate to high-level disconnects (e.g., neural signals, communication), what measures of coordination are best suited to capture disconnects (e.g., synchrony-based, entropy-based), and what methods are most precise in detecting low-level transitions related to disconnects (e.g., linear or nonlinear). To gain a deeper insight into instances of ineffective team functioning in the form of disconnects, future studies should assess how different team attributes, such as team composition, size, and roles, affect the occurrence and handling of disconnects. Such studies could inform both team researchers and professionals regarding what kind of teams are more prone to experience and overlook ineffective team coordination, and what mix of team attributes would be optimal for mitigating disconnects. Finally, future research should obtain a more complete understanding of the underlying nature of response delays following disconnects. Such knowledge could be further incorporated in team training, to attempt the optimization of adequate and timely responses to disconnects. Regardless of the nature of response delays, every second can be crucial in medical crises, emphasizing the importance of a robust in situ feedback system regarding inefficient team coordination, which may prevent long response delays.
Conclusion
In the current study, we found that transitions in low-level team coordination can identify high-level disconnects, as well as shifts in tasks or task phases. In addition, our results showed that teams typically address disconnects with explicit coordination but with considerable delays. Our study contributes important insights to the scarce research regarding high-level disconnects, especially in the context of acute health care. Future research can build on these findings by assessing more precise disconnect identification using low-level coordination metrics and gaining deeper insight into delayed disconnect responses in terms of high-level coordination. In sum, the current research advances our knowledge of disconnects toward ultimately enabling in situ support of teams in responding to and mitigating negative consequences of crisis situations.
Footnotes
Appendix A
Appendix B
Appendix C
Acknowledgements
We kindly thank Stefan Hasenoot and Lisanne Cornelis for their contributions to the manual annotation of high-level team coordination, prof. dr. Tanja Manser for the helpful discussions regarding high-level team coordination annotation frameworks, dr. Chris Emmery for his guidance regarding data analysis, Danny Struijk and Sophie Penning de Vries for their contributions to the data collection, and Maxima Medical Center Eindhoven and Spaarne Gasthuis for facilitating data collection.
ORCID iDs
Ethical Considerations
The current study was approved by the Ethical Review Board of Eindhoven University of Technology (ERB2022IEIS31).
Consent to Participate
Participants provided written informed consent to participate in the experiment, and for the inclusion of their data in analyses published in academic journals.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We gratefully acknowledge funding by the Dutch Research Council (NWO) as part of the NWO Complexity and Creative Industry: Grip on Transitions and Resilience program with project number 645.003.003 awarded to Dr. Travis J. Wiltshire and Prof. Dr. Josette M. P. Gevers.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.