Methodologies for the empirical measurement of situation awareness are reviewed, including a discussion of the advantages and disadvantages of each method and the potential limitations of the measures from a theoretical and practical viewpoint. Two studies are presented that investigate questions of validity and intrusiveness regarding a query-based technique. This technique requires that a simulation of the operational tasks be momentarily interrupted in order to query operators on their situation awareness. The results of the two studies indicate that the query technique is not intrusive on normal subject behavior during the trial and does not suffer from limitations of human memory, which provides an indication of empirical validity. The results of other validity studies regarding the technique are discussed along with recommendations for its use in measuring situation awareness in varied settings.
Get full access to this article
View all access options for this article.
References
1.
Bolstad, C. A., and Endsley, M. R. (1990). Single versus dual scale range display investigation (NOR DOC 90–90). Hawthorne, CA: Northrop Corporation.
2.
Carmody, M. A., and Gluckman, J. P. (1993). Task-specific effects of automation and automation failure on performance, workload and situational awareness. In R. S. Jensen and D. Neumeister, (Eds.), Proceedings of the Seventh International Symposium on Aviation Psychology (pp. 167–171. Columbus, OH: Ohio State University, Department of Aviation.
3.
Chase, W. G., and Simon, H. A. (1973). Perceptions in chess. Cognitive Psychology,4, 55–81.
4.
Cowan, N. (1988). Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information-processing system. Psychological Bulletin,104, 163–191.
5.
Endsley, M. R. (1987). SAGAT: A methodology for the measurement of situation awareness (NOR DOC 87–83). Hawthorne, CA: Northrop Corp.
6.
Endsley, M. R. (1988). Situation awareness global assessment technique (SAGAT). In Proceedings of the National Aerospace and Electronics Conference (NAECON) (pp. 789–795). New York: IEEE.
7.
Endsley, M. R. (1989a). Final report: Situation awareness in an advanced strategic mission (NOR DOC 89–32). Hawthorne, CA: Northrop Corp.
8.
Endsley, M. R. (1989b). Tactical simulation 3 test report: Addendum 1 situation awareness evaluations (81203033R). Hawthorne, CA: Northrop Corp.
9.
Endsley, M. R. (1990a). A methodology for the objective measurement of situation awareness. In Situational awareness in aerospace operations (AGARD-CP-478; pp. 1/1–1/9). Neuilly-Sur-Seine, France: NATO-Advisory Group for Aerospace Research and Development.
10.
Endsley, M. R. (1990b). Predictive utility of an objective measure of situation awareness. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 41–45). Santa Monica, CA: Human Factors and Ergonomics Society.
11.
Endsley, M. R. (1990c). Situation awareness global assessment technique (SAGAT): Air-to-air tactical version user guide (NOR DOC 89–58, rev A). Hawthorne, CA: Northrop Corp.
12.
Endsley, M. R. (1990d). Situation awareness in dynamic human decision making: Theory and measurement. Unpublished doctoral dissertation, University of Southern California. Los Angeles, CA.
13.
Endsley, M. R., and Rodgers, M. D. (1994). Situation awareness global assessment technique (SAGAT): En route air traffic control version user's guide (Draft). Lubbock: Texas Tech University.
14.
Fracker, M. L. (1990). Attention gradients in situation awareness. In Situational awareness in aerospace operations AGARD-CP-478; pp. 6/1–6/10). Neuilly-Sur-Seine, France: NATO-Advisory Group for Aerospace Research and Development.
15.
Herrmann, D. J. (1984). Questionnaires about memory. In J. E. Harris and P. E. Morris, (Eds.), Everyday memory, action and absent-mindedness (pp. 133–151. London: Academic.
16.
Hogg, D. N., Torralba, B., and Volden, F. S. (1993). A situation awareness methodology for the evaluation of process control systems: Studies of feasibility and the implication of use (1993–03–05). Storefjell, Norway: OECD Halden Reactor Project.
17.
Hughes, E. R., Hassoun, J. A., and Ward, F. (1990). Advanced tactical fighter (ATF) simulation support: Final report (CSEF-TR-ATF-90–91). Wright-Patterson Air Force Base, OH: Air Force Systems Command, Aeronautical Systems Division.
18.
Jacoby, L. L., and Dallas, M. (1981). On the relationship between autobiographical memory and perceptual learning. Journal of Experimental Psychology: General,110, 306–340.
19.
Kellog, R. T. (1980). Is conscious attention necessary for longterm storage?. Journal of Experimental Psychology: Human Learning and Memory,6, 379–390.
20.
Manktelow, K., and Jones, J. (1987). Principles from the psychology of thinking and mental models. In M. M. Gardiner and B. Christie, (Eds.), Applying cognitive psychology to user-interface design (pp. 83–117. Chichester, England: Wiley.
21.
Marshak, W. P., Kuperman, G., Ramsey, E. G., and Wilson, D. (1987). Situational awareness in map displays. In Proceedings of the Human Factors Society 31st Annual Meeting (pp. 533–535). Santa Monica, CA: Human Factors and Ergonomics Society.
22.
McDonnell Douglas Aircraft Corporation. (1982). Advanced medium-range air-to-air missile operational utility evaluation (AMRAAM OUE) test final report.. St. Louis, MO: Author.
23.
Mogford, R. H., and Tansley, B. W. (1991). The importance of the air traffic controller's mental model.Presented at the Human Factors Society of Canada Annual Meeting, Canada.
24.
Morton, J. (1969). Interaction of information in word recognition.Psychological Review,76,165–178.
25.
Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review,84, 231–259.
26.
Norman, D. A. (1968). Towards a theory of memory and attention. Psychological Review,75, 522–536.
27.
Northrop Corporation. (1988). Tactical simulation 2 test report: Addendum 1 situation awareness test results.. Hawthorne, CA: Author.
28.
Pew, R. W. (1991). Defining and measuring situation awareness in the commercial aircraft cockpit. In Proceedings of the Conference on Challenges in Aviation Human Factors: The National Plan (pp. 30–31). Washington, DC: American Institute of Aeronautics and Astronautics.
29.
Sarter, N. B., and Woods, D. D. (1991). Situation awareness: A critical but ill-defined phenomenon. International Journal of Aviation Psychology,1, 45–57.
30.
Selcon, S. J., and Taylor, R. M. (1990). Evaluation of the situational awareness rating technique (SART) as a tool for aircrew systems design. In Situational awareness in aerospace operations (AGARD-CP-478; pp. 5/1–5/8). Neuilly-Sur-Seine, France: NATO-Advisory Group for Aerospace Research and Development.
31.
Sheehy, E. J., Davey, E. C., Fiegel, T. T., and Guo, K. Q. (1993, April). Usability benchmark for CANDU annunciation--lessons learned.Presented at the ANS Topical Meeting on Nuclear Plant Instrumentation, Control and Man-Machine Interface Technology, Oak Ridge, TN.
32.
Smolensky, M. W. (1993). Toward the physiological measurement of situation awareness: The case for eye movement measurements. In Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting (p. 41). Santa Monica, CA: Human Factors and Ergonomics Society.
33.
Taylor, R. M. (1990). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. In Situational awareness in aerospace operations (AGARD-CP-478; pp. 3/1–3/17). Neuilly-Sur-Seine, France: NATO-Advisory Group for Aerospace Research and Development.
34.
Tulving, E. (1985). How many memory systems are there?. American Psychologist,40, 385–398.
35.
Venturino, M., Hamilton, W. L., and Dvorchak, S. R. (1990). Performance-based measures of merit for tactical situation awareness. In Situation awareness in aerospace operations (AGARD-CP-478; pp. 4/1–4/5). Neuilly-Sur-Seine, France: NATO-Advisory Group for Aerospace Research and Development.
36.
Vidulich, M. A. (1989). The use of judgment matrices in subjective workload assessment: The subjective workload dominance (SWORD) technique. In Proceedings of the Human Factors Society 33rd Annual Meeting (pp. 1406–1410). Santa Monica, CA: Human Factors and Ergonomics Society.
