An analysis of query-based measures for situation awareness was conducted, focusing on the effect of sampling from among the population of possible queries and the assumption that all queries are equally likely to be answered correctly across the population of participants. The analysis, which utilized Monte Carlo simulations, demonstrate that sampling from among the possible queries reduces the power of statistical tests, and the assumption of equal likelihood may result in a misleading portrayal of the results.
Get full access to this article
View all access options for this article.
References
1.
BurnsC. M.SkraaningJ. G.JamiesonG. A.LauN.KwokJ.WelchR.. (2008). Evaluation of ecological interface design for nuclear process control: Situation awareness effects. Human Factors, 50, 663-679.
2.
CarettaT. R.PerryD. C.ReeM. J. (1996). Prediction of situational awareness in F-15 pilots. International Journal of Aviation Psychology, 6, 21-41.
3.
ChanceyE. T.BlissJ. P. (In Press). Unreliable information in infantry situation awareness: Improvement through game-based training. Simulation & Gaming.
4.
CochranW. G. (1968). Errors of measurement in statistics. Technometrics, 10, 637-666.
5.
DursoF. T.DattelA. R. (2004). SPAM: The real-time assessment of SA. In BanburyS. (Ed.), A Cognitive Approach to Situation Awareness: Theory, Measures, and Application (pp. 137-154). London: Ashgate.
6.
EndsleyM. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37, 65-84.
7.
GronlundS. D.OhrtD. D.DoughertyM. R. P.PerryJ. L.ManningC. A. (1998). Role of memory in air traffic control. Journal of Experimental Psychology: Applied, 4(3), 263-280.
8.
HaussY.EyferthK. (2003). Securing future ATM-concepts’ safety by measuring situation awareness in ATC. Aerospace Science and Technology, 7, 417-427.
9.
LancasterJ. A.CasaliJ. G. (2008). Investigating pilot performance using mixed-modality simulated data link. Human Factors, 50, 183-193.
10.
LiuY.SalvendyG. (2008). Effects of measurement errors on psychometric measurements inergonomics studies: Implications for correlations, ANOVA, linear regression, factor analysis, and linear discriminant analysis. Ergonomics, 52, 499-511.
11.
ManzeyD.LuzM.MuellerS.DietzA.MeixensbergerJ.StraussG. (2011). Automation in surgery: The impact of navigated-control assistance of performance, workload, situation awareness, and acquisition of surgical skills. Human Factors, 53, 584-599.
12.
PritchettA. R.HansmanR. J.JohnsonE. N. (1995). Use of testable responses for performance-based measurement of situation awareness. In EndsleyM. R.GarlandD. J. (Eds.), Experimental Analysis and Measurement of Situation Awareness. Daytona Beach, FL: Embry-Riddle Press.
13.
SalmonP. M.StantonN. A.WalkerG.GreenD. (2006). Situation awareness measurement: A review of applicability for C4i environments. Applied Ergonomics, 37(2), 225-238.
14.
ScholtzJ.AntonishekB.YoungJ. (2004, 5 - 8 January). Evaluation of a human-robot interface: development of a situational awareness methodology. Paper presented at the 37th Annual Hawaii International Conference on System Sciences, Big Island, HI.
15.
SulistyawatiK.WickensC. D.ChuiY. P. (2011). Prediction in situation awareness: Confidence bias and underlying cognitive abilities. The International Journal of Aviation Psychology, 21, 153-174.
16.
TaylorR. M. (1989, October). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. Paper presented at the NATO AGARD conference on situational awareness in aerospace operations, Springfield, VA.
