Critical human factors challenges, including some that are unique to the cyber domain, demand technological solutions to aid operators performing cyber missions. One performance component of particular interest is measuring the situation awareness (SA) of human operators monitoring cyber events (herein called cyber SA). During a literature review, this paper’s authors found several attempts to develop and test measures of cyber SA, but none of the papers presented a framework integrating a formal definition of SA into the cyber domain. By crossing the US Army’s five-plane model of cyberspace operations with Endsley’s three-level model of SA, a formal framework is herein proposed. Using this framework, researchers can develop probe questions suitable for any cyber mission, and their results can be used to measure cyber SA. A sample application of this framework is provided, and future directions are proposed.
Get full access to this article
View all access options for this article.
References
1.
BrynielssonJ.FrankeU.VargaS. (2016). Cyber Situational Awareness Testing. In Akhgar Babak & B. and Brewster (Eds.), Combatting Cybercrime and Cyberterrorism: Challenges, Trends and Priorities (pp. 209–233). Springer Publishing.
2.
DursoF. T.HackworthC. A.TruittT. R.CrutchfieldJ.NikolicD.ManningC. A. (1998). Situation awareness as a predictor of performance for en route air traffic controllers. Air Traffic Control Quarterly, 6(1), 1–20.
3.
EndsleyM. R. (1988). Situation awareness global assessment technique (SAGAT). Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, 789–795.
4.
EndsleyM. R. (1995). Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors Journal37(1), 32–64. 10.1518/001872095779049543.
5.
EndsleyM. R. (1996). Situation awareness measurement in test and evaluation. Handbook of Human Factors Testing and Evaluation, 159–180.
6.
EndsleyM. R.SelconS. J.HardimanT. D.CroftD. G. (1998). A comparative analysis of SAGAT and SART for evaluations of situation awareness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(1), 82–86.
7.
EvangelopoulouM.JohnsonC. W. (2015). Empirical framework for situation awareness measurement techniques in network defense. 2015 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), 1–4.
8.
GiacobeN. A. (2013). A Picture is Worth a Thousand Alerts. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57(1), 172–176.
9.
GiacobeN. A.McNeeseM. D.MancusoV. F.MinotraD. (2013). Capturing human cognition in cyber-security simulations with NETS. 2013 IEEE International Conference on Intelligence and Security Informatics: Big Data, Emergent Threats, and Decision-Making in Security Informatics, 0525, 284-288.
LifP.GranasenM.SommestadT. (2017). Development and validation of technique to measure cyber situation awareness. 2017 International Conference On Cyber Situational Awareness, Data Analytics And Assessment, Cyber SA 2017.
12.
MahoneyS.RothE.SteinkeK.PfautzJ.WuC.FarryM. (2010). A Cognitive Task Analysis for Cyber Situational Awareness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 54, 279–283.
13.
MalviyaA.FinkG. A.SegoL.Endicott-PopovskyB. (2011). Situational Awareness as a Measure of Performance in Cyber Security Collaborative Work. Eighth International Conference on Information Technology: New Generations, 937–942.
14.
McGuinnessB. (2004). Quantitative analysis of situational awareness (QUASA): Applying signal detection theory to true/false probes and self-ratings. BAE Systems Advanced Technology Centre, Bristol, United Kingdom.
15.
PaulC. L.WhitleyK. (2013). A Taxonomy of Cyber Awareness Questions for the User-Centered Design of Cyber Situation Awareness. HAS/HCII 2013, 8030, 145–154.
16.
RajivanP.CookeN. (2017). Impact of team collaboration on cybersecurity situational awareness. In Theory and Models for Cyber Situation Awareness (pp. 203–226). Springer.
17.
RaymondD.CrossT.ContiG.NowatkowskiM. (2014). Key terrain in cyberspace: Seeking the high ground. 6th International Conference on Cyber Conflict, 287–300.
18.
RegnerR.WakeleeJ.BradshawJ.HitesL. (2009). Steps in writing an effective Master Scenario Events List. Journal of Emergency Management, 7(6).
19.
RousseauR.TremblayS.BanburyS.BretonR.GuitouniA. (2010). The role of metacognition in the relationship between objective and subjective measures of situation awareness. Theoretical Issues in Ergonomics Science, 11(1-2), 119–130.
20.
SalmonP. M.StantonN. A.WalkerG. H.JenkinsD.LadvaD.RaffertyL.YoungM. (2009). Measuring Situation Awareness in complex systems: Comparison ofmeasures study. International Journal of Industrial Ergonomic, 39(3), 490–500.
21.
SalmonP.StantonN.WalkerG.GreenD. (2006). Situation awareness measurement: A review of applicability for C4i environments. Applied ergonomics, 37(2), 225–238.
22.
TaylorR. M. (1990). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. In Situational Awareness in Aerospace Operations (AGARD-CP478) (pp. 3/1-3/17). Neuilly Sur Seine, France: NATO AGARD.
23.
U.S. Army. (2010). Cyberspace Operations Concept Capability Plan 2016-2028.
24.
U.S. Fleet Cyber Command. (2020) U.S. Fleet Cyber Command/U.S. Tenth Fleet Strategic Plan 2020-2025.
