Assessing Team Situation Awareness in the Field: A Comparison of Measurement Approaches

Abstract

In this paper we describe a field study conducted with trained Navy teams that was designed to assess the impact of an advanced watchstation on team performance. One of our primary hypotheses was that the advanced display technologies incorporated in this watchstation would support superior situation awareness (SA) within the Navy teams, when compared to the SA supported by their current watchstations. We attempted to use traditional probing techniques to assess team SA, but encountered several difficulties. On the other hand, by assessing the frequency of different classes of behaviors across objects with different levels of tactical significance, we were able to find a strong performance pattern that clearly supported our hypothesis. After explaining our approach and results, we briefly discuss implications for other efforts attempting to assess SA in field settings.

Get full access to this article

View all access options for this article.

References

Dwyer

D. J.

(1992) An index for measuring naval team performance. Proceedings of the Human Factors Society 36th Annual Meeting (pp. 1356–1360). Santa Monica, CA; Human Factors and Ergonomics Society.

Endsley

M. R.

(1988). Design and evaluation for situation awareness enhancement. Proceedings of the 32nd Annual Meeting of the Human Factors and Ergonomics Society(pp.97–101). Santa Monica, CA: Human Factors and Ergonomics Society.

Endsley

M. R.

(1995). Measurement of situation awareness. Human Factors, 37(1) 65–84.

Endsley

M. R.

(1997) The role of situation awareness in naturalistic decision making. In Zsambok

C. E.

Klein

(Eds.) Naturalistic Decision Making.(pp. 269–284). Mahwah, NJ: Lawrence Erlbaum.

Johnston

J. H.

Cannon-Bowers

J. A.

Smith-Jentsch

K. A

(1995) Event based performance measurement system for shipboard command teams. Proceedings of the 1st International Symposium on Command and Control Research and Technology, Washington, D. C. The Center for Advanced Command and Technology, pp. 274–276.

Pew

R. W.

(1995). The state of situation awareness measurement: Circa 1995. In Garland

D. J.

Endsley

M. R.

(Eds.) Experimental Analysis and Measurement of Situation Awareness: Proceedings of An International Conference (pp. 7–16). Daytona Beach FL: Embry-Riddle Aeronautical University Press.