A Vision Inference Action Model for Shared Situational Awareness in Autonomous Agents: Approaches,Challenges,& Future Directions

Abstract

Although there has been a recent increase of research investment in Human Autonomy Teams (HATs), many significant technical challenges and research gaps persist. Specifically, recent reports indicate the need for shared situational awareness within HATs. Shared situational awareness is a necessary precursor for essential team mechanisms, such as communication, coordination, cooperation, cohesion, and trust. Various modeling techniques have approached agent situational/shared awareness; however, these approaches have been limited to simulation environments. In order to achieve HATs in real-world contexts, agents will need to leverage autonomous perception (e.g., vision, audition, and task knowledge) to gain situational awareness of their environment and teammates. In this paper, we discuss approaches toward agent shared situational awareness, challenges/limitations, and review our efforts toward developing shared situational awareness within our Vision Inference Action (VIA) model and ideas for future research.