Abstract
Technological advances are increasing the complexity of, interactivity with, and cognitive demands on, human agents, particularly when AI is introduced. If we are to successfully reduce cognitive workload (CWL), we must adjust how we conceptualize cognition in manned-unmanned teaming environments. Towards this end, we draw from cognitive science theory, and the idea that the processes and products of cognition are extended across natural and artificial cognitive systems and embedded within a socio-technical environment. This necessitates a consideration of the system and environment as the same cognitive unit as the operator. We take a cognitive systems perspective and focus on the role of interaction and interdependence of human-machine teaming to provide insights for mitigating workload by extending cognition via scaffolding and offloading. Through this application of extended cognition theory, we offer a multi-disciplinarily informed approach, allowing us to provide a set of theoretically grounded research questions for understanding and improving workload.
Keywords
Introduction
Technological advances are increasing the complexity of interactivity with, and cognitive demands on, human agents. This trend is particularly evident in military aviation environments. Pilots face increasing cognitive demands as their operational environment extends beyond the cockpit and their cognitive processing is integrated with sensor and effector technologies. For example, aviation platforms outfitted with intelligent unmanned aerial vehicles are a new form of complex collaboration, where human and machine coordinate cognition and behavior to dynamically plan, replan, and adapt. If we are to successfully reduce cognitive workload, we must adjust how we conceptualize cognition in human-machine teaming environments. Following Risko and Dunn (2015), we suggest this requires human-machine systems must “adaptively integrate internal with external processes [to be] a successful cognitive agent in a complex environment” (p. 61).
In this paper, we elaborate on these ideas by drawing from a recent cognitive task analyses (CTA) of a specific complex operational environment, next generation Future Vertical Lift (Roth et al., 2022). From that, we focus on a set of concepts relevant to manned-unmanned teaming. For example, in the context of team cognition, situation awareness can be seen as critical to operators managing attention to, and awareness of their system’s state as well as the demands on their teammates during critical tasks (e.g., Cak et al., 2019). Related, decision making requires a set of cognitive processes where the operators take into account information from their system, their environment, and each other, to develop a course of action (Roth et al., 2022). From this conceptual base we can identify cognitive requirements and their roles in manned-unmanned teaming contexts. Towards this end, in an environment filled with cognitive artifacts, or artificial devices “designed to maintain, display, or operate upon information in order to serve a representational function” (Norman, 1991, p. 17) we can extend the scope of this CTA alongside our conceptions of cognition. Specifically, because pilots carry out missions in tandem with increasingly intelligent technologies, their cognitive processes are both internal and “spread across task and environmental artifacts, as well as team members” (Fiore & Wiltshire, 2016, p. 6; see also Hutchins, 1995).
With that as the operational foundation, we suggest that cognitive science theory, especially 4E cognitive theory, can effectively inform the pursuit to mitigate workload in the FVL environment. Modern cognitive science theory is shifting the perspective on cognition from being individual-centric to acknowledging and encompassing, for example, the dynamic(s) present in human-machine teams. More broadly construed, these human-machines teams are representative of what can be considered our modern day cognitive ecosystem given the technologically-advanced and somewhat dependent world we live in. And due to this, it is becoming more common the opinion that “a promising direction for cognitive science is the study of augmented intelligence, or the way social and technological systems interact with and extend individual cognition” (Dubova et al., 2022).
Within cognitive science, the 4E approach views cognition as extended, embedded, enactive, and embodied (Anderson et al., 2019). In using this to analyze the cognitive requirements outlined in Roth et al. (2022), we integrate concepts such as offloading and extended cognition. We pursue a cognitive systems perspective, emphasizing the role of interaction and interdependence of human-machine teaming to provide insights for efficiently reducing pilots’ cognitive workload.
Theoretically, we are driven by a set of broad questions:
(1) What is the relationship between extended cognition and human-machine systems?
(2) What cognitive concepts best lend themselves to the FVL environment?
Through this application of extended cognition, we can then offer a multi-disciplinarily informed approach, allowing us to provide recommendations for answering these questions, and meeting our goal(s).
Developing Research Questions
Cognitive requirements of the pilot-machine system are immense, dynamic and potentially unexpected. From pre-mission preparations, to in-mission replanning, and post-mission evaluations, this system, or team, exhibits extreme complexity through its interactions. To address this, we begin by highlighting key requirements such as replanning and cognitive workload. Our 4E cognitive approach draws from an understanding and application of cognitive offloading when conceptualizing the processes and products of cognition as extended across natural and artificial cognitive systems, and embedded within a socio-technical environment (Bocanegra et al., 2019; Fiore et al., 2010; Fiore & Wiltshire, 2016; Hollnagel, 2001).
The importance of 4E cognitive theory to our approach cannot be understated. As enacted, cognition unfolds in looping sensorimotor interactions between an active embodied organism and its environment. As an individual (e.g., pilot) acts, they are effectively “carrying out” cognition in a space. This “space” or environment is essential to our conception of cognition as being embedded. Intelligent thought and action are regularly, and perhaps sometimes necessarily, causally dependent on the bodily exploitation of certain environmental props or scaffolds.
Placing this in an applied context we can understand that a pilot does not physically nor mentally exist/behave in a vacuum. Instead they regularly make use of their cockpit and its affordances in order to enact cognition. In doing so, it is not uncommon for the agent in question to have their psychological states and processes shaped, in fundamental ways, by non-neural bodily factors. This emphasizes their cognition as being embodied, with pilots’ bodily, agential actions grounding their cognition in the environment. As extended, cognition is considered in such a way that the material vehicles realizing thinking and thoughts are spatially distributed over brain, body and world (Anderson et al., 2019). For example, when a pilot is using an Integrated Cueing Environment (ICE) in combination with multi-modal cuing to land, visuals are overlayed onto a display, tactile signals transmit important information, and the demand on the pilot to use spatio-visual reasoning is greatly reduced, effectively extending cognition across the pilot, system, and environment. These four “E”s serve as fundamentals not only for understanding cognition, also for studying and in turn, analyzing it. Essentially, 4E cognition as extended, or distributed, assists us in more accurately studying pilots’ cognition and the pilot-system-environment relationship in a way that allows for effective real-world insights.
Through incorporation of this framework, we more specifically apply an understanding of two key “techniques”—offloading and scaffolding. Offloading is present when an individual replaces what was previously internal processing and in turn frees up cognitive resources for other processes. For example, the aforementioned ICE system that replaces the need for the pilot to visualize the specifics of landing either in the real-world and/or on a map by instead overlaying relevant information on a display, offloading the internal processing onto the system and environment. On the other hand is scaffolding, which refers to externalizations that directly support processes by helping to mediate and support the interaction between individual and task and/or team-level cognitive activity (Fiore & Wiltshire, 2016). In the landing scenario, after offloading occurs via visualization of a flight path, the pilot’s decision-making is then scaffolded by the provision of path options. Both of these techniques help to manage cognitive workload in any given environment, while not removing the requirement, for example of the pilot, to ultimately decide and act.
To provide initial insight, from this theoretical application we can focus on the aforementioned needs to make recommendations for research and analysis. This includes, but is not limited to, the following representative recommendations:
Recommendation 1: Workload should be analyzed from a fully inclusive
Recommendation 2: Offloading techniques should be studied with varied degrees of accessibility to examine how cognitive efficiency varies
Recommendation 3: Analyses of
Recommendation 1 emphasizes the necessity to shift our focus to consider the ways in which an entire timeline can be of importance. Planning is intertwined with the way in which the mission itself unfolds, just as debriefing after is informed by the mission and consequently informs new developments. Recommendation 2 aims to inform the design side of things. For example, assessing agents’ proclivities toward offloading by imposing requirements to do so, not allowing them to do so, or having it be optional, are manipulations that directly inform those looking to effectively implement these features. Recommendation 3 supports the integration of prior work to assess the efficacy of, for example, offloading techniques in real-world cognitive contexts. By consideration of these recommendations we are able to develop and test theory-based guidance on mitigating workload, and develop models of workload applicable to manned-unmanned teaming environments.
By integrating theoretical developments from the cognitive sciences to the existing landscape of pilot cognitive workload analyses, we are able to generate novel understandings of how to approach a reduction of cognitive workload. Through this application we find that, for example, in regard to replanning, what is less understood is the inherently integrated information dynamics between human and machine. Specifically, it is the relationship between pilot-system-environment where replanning takes place. As such, this should not be viewed as occurring in either part separately or even sequentially. This finding, then, informs research design. In regard to integrative design that mirrors proper contexts, it suggests that testbeds need flexibility in methods for offloading information and decision-making, while also testing for extremes in accessibility. Further, it helps us show how an understanding of workload requires a testbed that mimics the varied ways workload occurs and/or evolves (e.g., dynamics, timing, distribution, buildup, decrease).
Extended cognition principles, though effective on their own, are immensely successful when synchronously applied to cognitive frameworks as opposed to features—in this case overlaying workload, offloading, and replanning analyses. While focusing on the ways in which physical and cognitive tasks can present themselves in workload, studies tend to focus on ways to measure workload instead of analyzing ways to manage and more importantly, effectively reduce it. This, as we first argued and through our approach show, necessitates a consideration of the system and environment as one in the same cognitive unit as the pilot themselves.
Conclusions
Whereas traditional studies of workload will view system, environment, and person independently, we view them as something occurring across an extended cognitive system. Applying offloading and other principles onto the mission environment provides the opportunity to better integrate solutions within the cognitive system as a whole. This approach allows us to consider how 4E cognitive principles can be used both separately and integratively when considering cognitive workload. This, in turn, allows us to more precisely design experiments that provide a more context-representative mission testbed scenario. Being able to translate these findings properly and knowledgeably to the context in which they operate “ensures that we can design systems which foster the flow of information around the system,” (Sorensen et al., 2011, p. 469) in a practical and efficient manner.
In the pursuit of developing precisely-designed experiments we must keep in mind the goals of understanding the nature of cognitive workload, and properly integrating analyses of techniques (e.g., offloading) and pertinent requirements (e.g., replanning). In doing so we can support the aim to yield conclusions imperative to developing models of workload that are applicable to FVL environments. The creation of such models would then allow for a structured integration of theory and practice that serves to inform further research and development.
Future Directions
The advent of more sophisticated artificial intelligence capabilities adds an additional layer to this issue. Recent research in human-machine teaming shows how AI with social intelligence can enhance performance on complex tasks (Bendell et al., 2024). Others have similarly shown that human-AI teaming can lead to not only changes in process, but also improvements in work products (Newton et al., 2022). Although our paper did not specifically address artificial intelligence, its relevance as a form of extended cognition, as well as a technological intervention to complement the ideas proposed here, adds to the research needs for this line of inquiry. As a path forward we offer a set of overarching questions that will guide how to develop and test practical beneficial system changes:
(1) What implementations might aid pilots in handling dynamically changing workload throughout missions?
(2) What sort of relevant design principles can we create to assist the integration of results into real-world contexts?
(3) How can a cognitive systems perspective re-evaluate the role of pilot in relation to their systems environment?
(4) How do interventions based upon artificial intelligence, alter, either positively or negatively, workload experienced, or design implementations?
In sum, by drawing from a more wholly integrated extended cognitive theory on reducing cognitive workload, we can develop a research framework capable of testing system design features increasing operational effectiveness.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Writing of this paper was partially supported by funding from Lockheed Martin Corporation to the second author. The views and opinions contained in this article are the authors’ and should not be construed as official or as reflecting the views of the University of Central Florida or Lockheed Martin Corporation.
