Sage Journals: Discover world-class research

Abstract

To be effective, future combined joint operations will require distributed teams to share assets via the Transfer of Tactical Control (ToTC). The goal of this research was to develop and conduct a usability assessment of ToTC enabling interfaces within a UxV command and control simulation testbed. Three interfaces were developed: Enhanced Play Workbook, ToTC Request Tile, and Enhanced Vehicle Status Tile. These interfaces were evaluated by eleven participants with UxV (MQ-9 pilots/sensor operators) or battle management experience within a base defense scenario. Within the resource constrained scenario, participant completed just over half of the tasks and ToTC request rates were low overall. When UxV requests were performed by participants, their feedback suggested that the interfaces enhanced team situation awareness, but the request process could be further streamlined to support situations in which there might be a high rate of requests.

Keywords

usability evaluation transfer of tactical control ecological interface design

Introduction

The military relies on teams to successfully accomplish mission tasks on small and large scales (Goodwin et al., 2018). Communication breakdowns or inefficiencies in transferring information and resources within these teams can have dire consequences (Xiao et al., 1996). The Department of Defense’s recent emphasis on distributed teaming for future military engagements brings about unique challenges, as the risks for communication breakdowns and inefficiencies in distributed teams is heightened (DeCostanza et al., 2018; Wroblewski, 2005). While communication for any team is important, it is even more critical for a distributed team (Powell et al., 2004) due to the resource-constrained environments teammates are forced to work in (Cannon-Bowers et al., 1992). Additionally, distributed teams cannot interact face-to-face; instead, they rely on novel technology to communicate goals and information with non-collocated teammates (Anderson & Dossick, 2014; Cramton, 2001; Cramton & Orvis, 2003; Saunders & Ahuja, 2006). Without adequate tools to support distributed team communication and coordination, there is a heightened risk of mission failure. Therefore, distributed teams need interfaces that support shared situation awareness and that enable collaboration across distributed teammates.

In a previous effort to support distributed teams of heterogeneous unmanned vehicle (UxV) operators, a suite of interfaces was developed and implemented in the “Intelligent Multi-UxV Planner for Adaptive Collaborative/Control Technologies (IMPACT)” research testbed (Calhoun et al., 2017; Calhoun et al., 2018). IMPACT combines advancements in autonomy, intuitive interfaces, and decision support tools to facilitate command and control of multiple heterogeneous (air, ground, and sea surface) UxVs by an operator (or multiple operators) during a base defense mission (Draper et al., 2018). IMPACT’s task management and UxV allocation interfaces have previously been shown to be effective in supporting task completion and task sharing across a distributed team of UxV operators (Frost, 2019).

Distributed teams of UxV operators often engage in joint taskings using their own individually designated UxVs because current interoperability issues and doctrinal differences make sharing UxVs between team members difficult. Although challenging, the ability to conduct the Transfer of Tactical Control (ToTC) (i.e., share UxVs) is vital to future combined joint operations (Hupy, 2000). Specifically, distributed teams of UxV operators with a common mission objective, but individual taskings and UxVs, need to be able to efficiently conduct ToTC to ensure effective UxV assignment and overall mission success.

Interface Design to Support ToTC

To support efficient ToTC between distributed teams, three new or enhanced interfaces were developed and implemented within IMPACT: an Enhanced Play Workbook with teammate UxV allocation considerations, a new ToTC Request Tile with request notifications, and an Enhanced Vehicle Status Tile with teammate UxV awareness.

Enhanced Play Workbook

To support rapid tasking of UxVs, IMPACT operators communicate high-level objectives via a “play call” (e.g., search an area). In response, IMPACT provides operators with a ranked set of Courses of Action (COAs) to complete that objective. Each COA consists of a UxV allocation(s) as well as the associated route(s) (Behymer et al., 2017). The UxV allocation is completed by an intelligent agent that was developed using the Cognitively Enhanced Complex Event Processing framework (Hansen et al., 2016).

IMPACT’s Play Workbook supports play calling by summarizing the play details, constraints considered, and the recommended COA (detailed description provided in Calhoun et al., 2018) which can be reviewed and accepted by a user. After the play is accepted the UxV(s) is rerouted to complete the play and the workbook closes.

Previously, the intelligent agent’s pool of available UxVs for allocation was limited to a single operator’s UxVs. However, to leverage ToTC between distributed teammates, the pool of UxVs reasoned over needed to be expanded to include multiple operators’ UxVs when creating COAs. To support this functionality, the Play Workbook was enhanced to include two new tabs to allow the operator to specify which UxVs should be considered by the intelligent agent (shown in Figure 1 in the dotted box). By default, the “Owned” tab is selected, allowing the intelligent agent to only consider UxVs that the operator currently has control over. By changing the tab selection to “All” the intelligent agent will reason over all available UxVs, including those controlled by their teammates.

Figure 1.

Enhanced play workbook.

In this Enhanced Play Workbook, when an operator reviews a COA recommended by the intelligent agent, if the recommended UxV is already controlled by the operator the COA can be accepted as described above. However, if the allocated UxV is owned by another teammate, the button to accept the play changes to an arrow (shown in Figure 1 in the dotted circle), indicating a ToTC request needs to be created. Selecting the arrow icon opens the ToTC Request Tile.

Enhanced Vehicle Status Tile

IMPACT’s Vehicle Status Tile lists all the UxVs an operator controls along with status information (e.g., tasking, alerts, etc.). Each UxV is represented in a vehicle card that contains the UxV’s call sign and a color-coded vehicle symbol (white = available, colored = on play/tasked). UxV alerts (e.g., low fuel) are portrayed via a red highlighting.

The Enhanced Vehicle Status Tile (Figure 2) provides information about all UxVs across a team of operators and organizes UxVs into sections based on who has control. In Figure 2, the operator with the callsign “Maverick’s” UxVs are in the left bin and their teammate “Rooster’s” UxVs are in the right. To access the ToTC Request Tile for a specific UxV, a button was added on the left side of all UxV symbols. When a UxV has a pending request, a notification appears on the right of the UxV symbol (shown for Tiger-20 in Figure 2). The operator can select the request notification to view the ToTC Request Tile and approve/deny the request.

Figure 2.

Enhanced vehicle status tile.

ToTC Request Tile

The ToTC Request Tile interface (Figure 3) was designed to support ToTC by facilitating the sharing of necessary information between teammates. The operator requesting control of a UxV can specify the details of the request including which UxV is being requested, what the UxV will be doing, how long it will be in use, and if it will be returned upon task completion. Additionally, a comment section is available for unstructured text to communicate additional information to their teammate.

Figure 3.

ToTC request tile.

The ToTC Request Tile is accessible from both the Enhanced Play Workbook and the Enhanced Vehicle Status Tile. When accessing the ToTC Request Tile from the Enhanced Play Workbook, information regarding which UxV is being requested and how the UxV will be used is pre-populated based upon the specified play. Accessing the ToTC Request Tile from the Enhanced Vehicle Status Tile allows an operator to request a specific UxV from a teammate. Additionally, the operator can select a UxV they have control over and use the ToTC Request Tile to initiate a transfer of that UxV to a teammate.

The operator receiving the request (to either transfer one of their UxVs or to receive a UxV from their teammate) can review the information provided in the ToTC request and approve/deny the ToTC request. If the request is accepted the UxV is transferred, and its vehicle card moves to the corresponding teammate’s bin in the Enhanced Vehicle Status Tile. If the request is denied the UxV remains with the current operator.

Methods

A usability assessment was conducted to determine the utility of the individual ToTC interfaces and gain insight into the overall ToTC process. Each participant completed a trial in which they worked with a confederate teammate to complete to a simulated operational base defense scenario that required using the interfaces to transfer UxVs and/or tasks. Objective performance measures were collected to understand the use of the ToTC interfaces to complete tasks. Subjective data were collected through surveys and a debrief interview to evaluate features of the interfaces and collect feedback, which were analyzed using a grounded theory approach (Braun & Clarke, 2006).

Participants

Eleven participants with either UxV (MQ-9 pilots/sensor operators) or battle management (Air Battle Managers) experience were recruited for this study. Nine participants had direct UxV control experience and four of those participants had experience with ToTC during crew exchange or take-off/landing procedures. There were nine men and two women participants between the ages of 22 to 43 (M = 33, SD = 7.9).

Simulation Testbed

The testbed consisted of two IMPACT operator stations and a Test Operator Console (Figure 4). The stations were set up such that participants could not see each other or their screens. For more details of the apparatus, see Frost (2019).

Figure 4.

IMPACT multiple system setup including a test operator console (TOC).

Trial

Participants were asked to fill the role of a base defense operator working with another distributed operator (confederate). The trial required the team to work together to manage UxVs and complete a series of mixed priority tasks in response to various events within a simulated base defense scenario that was similar to those used in previous studies (Frost, 2019; Frost et al., 2021). Each teammate started with control of four different UxVs (two air, one ground, and one sea surface). The tasks and limited number of UxVs with varying capabilities were chosen to create resource constrained situations that necessitated task or UxV transfers or the shedding of low priority tasks.

Specifically, the team’s objective was to complete all tasks accurately (allocated UxV had the correct capability to complete the task). Each task in the scenario had UxV requirements to consider for allocation, which were communicated in the task description. Participants were instructed that to complete the task accurately they must select the UxV that best matches the task requirements (even if that meant requesting a teammate’s UxV or transferring the task to their teammate). Thus, to successfully meet the objective, teammates had to transfer tasks and UxVs. To conduct ToTC participants would need to utilize the three ToTC interfaces described above and could coordinate further with IMPACT’s chat tile. To transfer tasks participants used an Enhanced Task Manager developed and evaluated in previous research (Frost e tal., 2022).

Procedure

Participants were trained on the IMPACT system and the three ToTC interfaces through hands on instructions while walking through a self-paced trial that allowed participants to familiarize themselves with the system and the various task types. Participants then worked with a confederate teammate to complete a training trial to ensure they were able to successfully complete a scenario using all the interfaces, with researcher guidance only provided upon request. Once participants successfully completed a training trial and stated that they were comfortable using the system on their own, they worked with the confederate teammate to complete a 30-min capstone trial. After the capstone, participants filled out a questionnaire and were debriefed. Total session time was approximately 3.5 hr per participant.

Measures

During the capstone trial objective measures regarding performance and transfers were recorded, including: whether the task was consider Completed (correct UxV assigned to task) or Not Completed (no/incorrect UxV assigned to task), number of times the participant initiated a UxV transfer, number of tasks the participant transferred to their teammate, and the interface used to initiate ToTC requests. Post-capstone trial questionnaires included the NASA-TLX (Hart & Staveland, 1988), System Usability Scale (SUS) (Brooke, 1996), and Likert-scale ratings of interface features.

Results

Task Completion and Transfers

Participants completed 83 (57.6%) of the 144 tasks assigned to them. This rate of completion was anticipated given the high density of events within the scenario and the limited number of UxVs across teammates. Of these 83 completed tasks, participants completed 52 of them without transferring a UxV or the task. Of the remaining 31 completed tasks, participants used TOTC for 16 tasks, used a task transfer for six tasks, and used both TOTC and task transfer for nine tasks. Table 1 below describes the distribution of task completion and transfer data for tasks.

Table 1.

Distribution of Tasks and Transfers.

Value	Not completed	Completed 83 (57.6%)				Total
Value	Not completed	No transfer	ToTC	Task transfer	Both ToTC and task	Total
Count (%)	61 (42.4%)	52 (36.1%)	16 (11.1%)	6 (4.2%)	9 (6.3%)	144 (100%)

Of the 25 times ToTC was conducted, four times (16%) participants initiated the request from the Enhanced Workbook, eight times (32%) participants initiated the request from the Enhanced Vehicle Status Tile, eight times (32%) participants used the IMPACT chat tile to coordinate a transfer, and five times (20%) participants used a combination of the Enhanced Vehicle Status Tile and the chat tile. In the cases where a combination of request methods was used, participants were observed using chat to coordinate UxV availability and communicate intent to their teammate prior to issuing a formal request within the Enhanced Vehicle Status Tile. Not only was the Enhanced Vehicle Status Tile used more often to initiate ToTC requests, but also it was used more diversely regarding the types of requests that were initiated. Of the Enhanced Vehicle Status Tile requests, 42.9% were pull requests (participant requesting a UxV be transferred from the teammate to them) and 57.1% were push requests (participant requesting a UxV be transferred from them to the teammate). Conversely, all four workbook requests were pull requests, perhaps alluding to a bi-directionality issue.

Subjective Data

On a scale from one (poor performance) to seven (excellent performance), participants rated their self and team performance as average (M = 3.50, SD = 0.90, and M = 4.08, SD = 0.99, respectively). Participants expressed that they felt their teammate was more experienced and thus improved team performance.

Participants rated the difficulty they experienced in completing three aspects of the trial: completing tasks, allocating the correct UxV to a task, and coordinating a transfer. The mean ratings are shown in Figure 5. Participants rated coordinating ToTC or task transfer as the most difficult, and when asked in interviews, stated that despite having the ToTC Request Tile they additionally used the chat tile to fully articulate transfer needs and this was rather time consuming.

Figure 5.

Task difficulty ratings.

The system as a whole (ToTC interfaces integrated with IMPACT) received an acceptable SUS score (Bangor et al., 2009) with a grand mean of 73.30 (SD = 6.85). When providing ratings on a scale one (not at all) to seven (extremely), the ToTC interfaces were rated, separately from IMPACT, as more than moderately useful (M = 5.17, SD = 1.53). When asked about the request notifications, participants rated their awareness as moderate (M = 4.67, SD = 1.44). The average NASA-TLX score (M = 46.63, SD = 15.13) was somewhat high (Prabaswari et al., 2019).

Debrief Interview

The participants’ comments from the debrief interview following the execution of the capstone trial provide further insight regarding team situation awareness and the transfer request process.

Team Situation Awareness

Participants stated that the ToTC interfaces supported team situation awareness by providing them visibility of their teammate’s status (e.g., taskings, available UxVs, etc.), thus allowing them to anticipate their teammate’s needs and reduce coordination and communication requirements. This claim is supported by the amount of push requests that were initiated by participants.

However, participants also stated that outside of the Enhanced Vehicle Status Tile, it was difficult to determine who had control of which UxV at a given time and suggested more saliently depicting this information across the system. Lastly, the participants’ reliance on the chat tile to coordinate transfer details introduced unexpected awareness issues. The chat tile utilized a dual tab design and could only show one source at a time, either tasking info or communication with their teammate. This type of design limited participants’ awareness of new chat messages at times. To remedy this issue, participants suggested that the chat tile be integrated with the ToTC request tile. This would solve the awareness issue as well as support operators in efficiently articulating all transfer needs from a single interface.

Transfer Request Process

To better understand participants’ approaches to the transfer process they were asked to elaborate on how they evaluated incoming requests and if they employed any type of strategy for initiating transfer requests. While some participants explained that they blindly accepted all incoming UxV requests, citing trust in their teammate and the ability to request a UxV back if needed. Others reported using the ToTC interfaces to confirm their teammate’s current UxV allocation and the associated tasks’ relative priority prior to accepting or denying the request.

As for initiating requests, participants described determining the relative priority of the task compared to the current tasking of the UxV in question prior to initiating the request. This cross check, facilitated by the task management interface (reference in Frost, 2019) and the Enhanced Vehicle Status Tile, helped to ensure requests were accepted by their teammate and minimized coordination time. Overall, participants stated they used a mix of task transfers and ToTC as they felt transferring UxVs took more steps to complete.

Discussion and Conclusions

Subjectively, participants rated the ToTC interfaces as useful, citing an increased awareness of team tasking and an enhanced ability to anticipate teammate requirements. The objective data suggests that the ToTC process and the supporting interfaces still need further refinement to be fully embraced by operators though, especially in situations requiring high rates of transfers.

Despite stating that ToTC transfers were cumbersome and required more steps than task transfers, participants used ToTC more often than task transfers to manage task requirements in the capstone trial. Unexpectedly, participants seemed to rely on the chat tile in addition to the ToTC request tile when initiating requests, with participants utilizing chat in ToTC requests 48% of the time. Future work should look at a ToTC request tile that is integrated with the chat tile, thus allowing for teammates to freely communicate back and forth regarding the details of a request within a single interface.

This usability assessment was an initial investigation into how operators, using a multi-UxV command and control simulation testbed, managed transferring UxVs and tasks with a distributed teammate in a highly complex, resource constrained base defense scenario. The findings drawn from the feedback on the ToTC and task sharing interfaces and processes provide valuable information and motivation for developing a more rigorous research approach aimed at enhancing distributed human teaming and overall mission effectiveness.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Anderson

Dossick

C. S.

(2014). Avatar-model interaction in virtual worlds improves distributed team collaboration through issue discovery. Computing in Civil and Building Engineering, 793–800. https://doi.org/10.1061/9780784413616.09

Bangor

Kortum

Miller

(2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4, 114–123.

Behymer

Rothwell

Ruff

Patzek

Calhoun

Draper

Lange

(2017). Initial evaluation of the intelligent multi-uxv planner with adaptive collaborative/control technologies (IMPACT). Defense Technical Information Center.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101.

Brooke. (1996). SUS: A ‘quick and dirty’ usability scale. In Jordan

Thomas

Weerdmeester

McClelland

, Usability evaluation in industry (pp. 189–194). London, UK: Taylor & Francis.

Calhoun

G. L.

Ruff

H. A.

Behymer

K. J.

Frost

E. M.

(2018). Human-autonomy teaming interface design considerations for multi-unmanned vehicle control. Theoretical Issues in Ergonomics Science, 19, 321–352.

Calhoun

G. L.

Ruff

H. A.

Behymer

K. J.

Mersch

E. M.

(2017). Operator-autonomy teaming interfaces to support multi-unmanned vehicle missions. Advances in human factors in robots and unmanned systems: Proceedings of the AHFE 2016 international conference on human factors in robots and unmanned systems, July 27–31 (pp. 113–126). Walt Disney World, FL: Springer International Publishing.

Cannon-Bowers

J. A.

Oser

Flanagan

D. L.

(1992). Work teams in industry: A selected review and proposed framework. In Swezey

R. W.

Salas

, Teams: Their training and performance (pp. 355–377). New York, NY: Ablex Publishing.

Cramton

C. D.

(2001). The mutual knowledge problem and its consequences for dispersed collaboration. Organization Science, 12, 346–371.

10.

Cramton

C. D.

Orvis

K. L.

(2003). Overcoming barriers to information sharing in virtual teams. Virtual teams that work: Creating conditions for virtual team effectiveness (pp. 214–230). San Francisco: Jossey-Bass.

11.

DeCostanza

A. H.

Marathe

A. R.

Bohannon

Evans

A. W.

Palazzolo

E. T.

Metcalfe

J. S.

McDowell

(2018). Enhancing human agent teaming with individualized, adaptive technologies: A discussion of critical scientific questions. Aberdeen Proving Ground, United States: US Army Research Laboratory.

12.

Draper

Rowe

Douglass

Calhoun

Spriggs

Kingston

Frost

(2018). Realizing autonomy via intelligent hybrid control: Adaptable autonomy for achieving UxV RSTA team decision superiority. WPAFB, OH: Air Force Research Laboratory 711th HPW.

13.

Frost . (2019). Creating a well-situated human-autonomy team: The effects of team structure. Dayton, OH: Doctoral dissertation, Wright State University.

14.

Frost

Ruff

Behymer

(2022). Lessons learned and future direction of a teaming interface for unmanned vehicle tasks. Human factors robots, drones and unmanned systems: proceedings of the AHFE 2022 international conference (pp. 86–91). New York, NY: Springer International Publishing.

15.

Frost

Ruff

Calhoun

(2021). Human-autonomy teaming for unmanned vehicle control: Examination of the role of operator individual differences. Advances in human factors in robots, drones and unmanned systems: Proceedings of the AHFE 2020 virtual conference on human factors in robots, drones and unmanned systems (pp. 17–23). Springer International Publishing.

16.

Goodwin

G. F.

Blacksmith

Coats

M. R.

(2018). The science of teams in the military: Contributions from over 60 years of research. American Psychologist, 73, 322.

17.

Hansen

Calhoun

Douglass

Evans

(2016). Courses of action display for multi-unmanned vehicle control: A multi-disciplinary approach (Technical Report FS-16-03). Arlington, VA: AAAI Press.

18.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in Psychology, 52, 139–183.

19.

Hupy

(2000). Tactical control of air mobilty forces in operation allied force: Is this the way things should be done? Maxwell AFB, AL: Air Command and Staff College.

20.

Powell

Piccoli

Ives

(2004). Virtual teams: A review of current literature and directions for future research. ACM SIGMIS Database: The DATABASE for Advances in Information Systems, 35, 6–36.

21.

Prabaswari

Basumerda

Utomo

(2019). The mental workload analysis of staff in study program of private educational organization. In Proceedings of the IOP conference series: Materials science and engineering. Bristol, UK: IOP Publishing.

22.

Saunders

C. S.

Ahuja

M. K.

(2006). Are all distributed teams the same? Differentiating between temporary and ongoing distributed teams. Small Group Research, 37, 662–700.

23.

Wroblewski

E. M.

(2005). Collaboration tools: Meeting interoperability requirements in today’s military. Naval Air Systems Command.

24.

Xiao

Hunter

W. A.

Mackenzie

C. F.

Jefferies

N. J.

Horst

R. L.

Group

(1996). Task complexity in emergency medical care and its implications for team coordination. Human Factors, 38, 636–645.

Interface Design to Support Transfer of Tactical Control (ToTC) in Distributed Teams

Abstract

Keywords

Introduction

Interface Design to Support ToTC

Enhanced Play Workbook

Enhanced Vehicle Status Tile

ToTC Request Tile

Methods

Participants

Simulation Testbed

Trial

Procedure

Measures

Results

Task Completion and Transfers

Subjective Data

Debrief Interview

Team Situation Awareness

Transfer Request Process

Discussion and Conclusions

Footnotes

Declaration of Conflicting Interests

Funding

References