Modern military operations demand coordinated activity across multiple domains, including land, air, sea, space, and cyberspace. While existing tools well support simulations and visualizations of physical-domain effects, cyber effects are less well instantiated in current training and decision-support environments. Although cybersecurity visualizations do not necessarily require geospatial representation of device locations, in this tactical context, threats and assets across all domains, including cyber, can be depicted together on a traditional two-dimensional (2D) common operating picture map. However, three-dimensional (3D) views better support tactical navigation and targeting by enhancing the apprehension of depth, elevation, aerial assets, and 3D urban and terrain features. In operational contexts, users can view the physical environment and additional information (e.g., device status) via augmented reality. Our XR Cyber Battlefield Effects Correlation and Simulation tool (CYBERCAST) enables interactive 3D visualization of a multi-domain battlespace with a dynamic simulated scenario involving cyber effects. The system visually alerts users to locations of affected devices and enables them to obtain more information via hover. To provide this XR experience, it was necessary to integrate outputs from three different simulation tools (OneSAF, CyberVAN, and CyberBOSS) in a first-of-its-kind visualization and simulation system.
RussellSAbdelzaherT.The internet of battlefield things: the next generation of command, control, communications and intelligence (C3I) decision-making. In: MILCOM 2018-2018 IEEE military communications conference (MILCOM), Los Angeles, CA, 29–31 October 2018, pp. 737–742. New York: IEEE.
2.
KimJGregoryTFreemanJ, et al. System-of-systems for remote situational awareness: Integrating unattended ground sensor systems with autonomous unmanned aerial system and android team awareness kit. In: 2022 International Conference on Unmanned Aircraft Systems (ICUAS), 21–24 June 2022, pp. 1418–1423. New York: IEEE.
StanikJKiedrowiczM.Integration of national cybersecurity standards into the geographic information systems in the context of critical infrastructure security management. GIS Odyssey J2025; 5: 111–121.
KimKYounJYoonS, et al. Study on cyber common operational picture framework for cyber situational awareness. Appl Sci2023; 13(4): 2331.
11.
JiSYJeongBKJeongDH.Evaluating visualization approaches to detect abnormal activities in network traffic data. Int J Inform Secur2021; 20(3): 331–345.
12.
NackEBastianND.Synthetic environments for the cyber domain. Cyber Defense Review2023; 8(2): 91–104.
13.
AkpanIJShankerM.A comparative evaluation of the effectiveness of virtual reality, 3D visualization and 2D visual interactive simulation: an exploratory meta-analysis. Simulation2019; 95(2): 145–170.
14.
JeongJChoiAKangD, et al. A comparative study of 2D vs. 3D chart visualizations in virtual reality: a comparative study of 2D vs. 3D chart visualizations. J Vis2024; 28(1): 239–253.
15.
AskTFKullmanKSütterlinS, et al. A 3D mixed reality visualization of network topology and activity results in better dyadic cyber team communication and cyber situational awareness. Front Big Data2023; 6: 1042783.
KullmanKBuchananLKomlodiA, et al. Mental model mapping method for cybersecurity. In: HCI for cybersecurity, privacy and trust: second international conference, HCI-CPT 2020, Held as part of the 22nd HCI international conference, HCII 2020, Copenhagen, Denmark, 19–24 July 2020. New York: Springer, 2020, pp. 458–470.
18.
KullmanKCowleyJBen-AsherN.Enhancing cyber defense situational awareness using 3D visualizations. In: Proceedings of the 13th international conference on cyber warfare and security ICCWS 2018: National Defense University, Washington, DC, Vol. 8. Washington, DC: Academic Conferences and Publishing International Limited, 2018, pp. 369–378.
HaleKSCampbellGRileyJ, et al. Augmented reality sandtable (ARES) impacts on learning. In: Proceedings of the human factors and ergonomics society annual meeting, Vol. 63, Los Angeles, CA. New York: Sage, 2019, pp. 2149–2153.
21.
BoyceMWThomsonRHCartwrightJK, et al. Enhancing military training using extended reality: a study of military tactics comprehension. Front Virt Real2022; 3: 754627.
22.
SummersJE.Simulation-based military training: an engineering approach to better addressing competing environmental, fiscal, and security concerns. J Washingt Acad Sci2012: 9–29.
23.
TanwarVAnandVAggarwalP, et al. Revolutionizing military training: a comprehensive review of tactical and technical training through augmented reality, virtual reality, and haptics. In: 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), Pune, India, 5–7 April. New York: IEEE, 2024, pp. 1–5.
GoldbergBOwensKGuptonK, et al. Forging competency and proficiency through the synthetic training environment with an experiential learning for readiness strategy. In: Interservice/industry training, simulation, and education conference (I/ITSEC), Orlando, FL, 29 November–2 December, 2021.
26.
TaylorGDeschampsATanakaA, et al. Augmented reality for tactical combat casualty care training. In: Augmented cognition 2018—lecture notes in artificial intelligence, Vol. 10916. New York: Springer, 2018, pp. 227–239.
DennisonMReardonCGregoryJ, et al. Creating a mixed reality common operating picture across C2 echelons for human-autonomy teams. In: Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-domain Operations, Vol. 1142. New York: SPIE, 2020, pp. 137–144.
31.
ZoccoALivatinoSDe PaolisLT.Stereoscopic-3D vision to improve situational awareness in military operations. In: International Conference on Augmented and Virtual Reality. New York: Springer, 2014, pp. 351–362.
32.
PykeABouchelleRUzhcaD.Out of sight but still in mind: making “Invisible” cyber threats more salient via concrete analogies. Human Factor Cybersecur2023: 20–27.
33.
UsbeckKGillenMLoyallJ, et al. improving situation awareness with the Android Team Awareness Kit (ATAK). In: Sensors, and command, control, communications, and intelligence (C3I) technologies for homeland security, defense, and law enforcement XIV, Vol. 9456. New York: SPIE, 2015, pp. 172–193.
