The purpose of this study was to evaluate the independent and joint effects of objective and subjective measures of situation awareness on team performance. We propose that objective and subjective measures are types of metacognitive experiences, with objective measures capturing actual situation knowledge and subjective measures capturing perceived situation knowledge. These metacognitive experiences are theorized to work in tandem with task confidence to predict performance. Data were collected from 72 three-person teams working in a simulated task environment of emergency management teams. Objective situation awareness, subjective situation awareness, and task confidence were each measured through the Situation Awareness Global Awareness Technique, the Mission Awareness Rating Scale, and collective efficacy, respectively. Results showed that all three metacognitive experiences interacted in predicting team performance, such that task confidence strengthened the relationship between actual situation knowledge and performance when the team’s perceived situation awareness was low and weakened the relationship when the team’s perceived situation awareness was high. These results suggest that it is always important to have the necessary actual situation knowledge to complete a task, but there are times in which aspects of awareness in relation to judgments or feelings should be low in order to create the necessary drive to regulate one’s behavior.
AguinisH.GottfredsonR. K. (2010). Best-practice recommendations for estimating interaction effects using moderated multiple regression. Journal of Organizational Behavior, 31, 776–786.
2.
AikenL. S.WestS. G. (1991). Multiple regression: Testing and interpreting interactions. Thousand Oaks, CA: Sage.
3.
BanduraA. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.
4.
BellB. S.KozlowskiS. W. J. (2008). Active learning: Effects of core training design elements on self-regulatory processes, learning, and adaptability. Journal of Applied Psychology, 93, 296–316.
5.
BlieseP. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations (pp. 349–381). San Francisco, CA: Jossey-Bass.
6.
ChanD. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246.
7.
CohenJ. (1992). A power primer. Psychological Bulletin, 112, 155–159.
8.
CookeN. J.KiekelP. A.HelmE. E. (2001). Measuring team knowledge during skill acquisition of a complex task. International Journal of Cognitive Ergonomics, 5, 297–315.
9.
DunningD.JohnsonK.EhrlingerJ.KrugerJ. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12, 83–87.
10.
EdwardsR. K.KellnerK. R.SistromC. L.MagyariE. J. (2003). Medical student self-assessment of performance on an obstetrics and gynecology clerkship. American Journal of Obstetrics and Gynecology, 188, 1078–1082.
11.
EfklidesA. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process?Educational Research Review, 1, 3–14.
12.
EfklidesA. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-and co-regulation. European Psychologist, 13, 277–287.
13.
EfklidesA. (2014). How does metacognition contribute to the regulation of learning? An integrative approach. Psychological Topics, 23, 1–30.
14.
EhrlingerJ.JohnsonK.BannerM.DunningD.KrugerJ. (2008). Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105, 98–121.
15.
EndsleyM. R. (1995a). Measurement of situation awareness in dynamic systems. Human Factors, 37, 65–84.
16.
EndsleyM. R. (1995b). Toward a theory of situation awareness in dynamic system. Human Factors, 37, 32–64.
17.
EndsleyM. R. (2000). Theoretical underpinnings of situation awareness: A critical review. In EndsleyM. R.GarlandD. J. (Eds.), Situation awareness analysis and measurement (pp. 3–32). London, UK: Lawrence Erlbaum.
18.
EndsleyM. R. (2015a). Final reflections: Situation awareness models and measures. Journal of Cognitive Engineering and Decision Making, 9, 101–111.
19.
EndsleyM. R. (2015b). Situation awareness misconceptions and misunderstandings. Journal of Cognitive Engineering and Decision Making, 9, 4–32.
20.
EndsleyM. R.GarlandD.J. (2000). Situation awareness analysis and measurement. Mahwah, NJ: Lawrence Erlbaum.
21.
EndsleyM. R.SelconS. J.HardimanT. D.CroftD. G. (1998). A comparative analysis of SAGAT and SART for evaluations of situation awareness. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting (pp. 82–86). Santa Monica, CA: Human Factors and Ergonomics Society.
22.
EndsleyM. R.SollenbergerR. L.NakataA.SteinE. S. (2000). Situation awareness in air traffic control: Enhanced displays for advanced operations. Atlantic City, NJ: Federal Aviation Administration.
23.
FlavellJ. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911.
24.
HamiltonK. (2009). The effect of team training on team mental model formation and team performance under routine and non-routine environmental conditions (Unpublished doctoral dissertation). Pennsylvania State University, University Park.
25.
HamiltonK.MancusoV.MinotraD.HoultR.MohammedS.McNeeseM.ParrA.DubeyG.McMillanE. (2010). Using the NeoCITIES 3.0 simulation to study and measure team cognition. In Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting (pp. 433–437). Santa Monica, CA: Human Factors and Ergonomics Society.
26.
HellarB. (2009). An investigation of data overload in team-based distributed cognition systems (Unpublished doctoral dissertation). Pennsylvania State University, University Park.
27.
HellarD. B.McNeeseM. (2010). NeoCITIES: A simulated command and control task environment for experimental research. In Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting (pp. 1027–1031). Santa Monica, CA: Human Factors and Ergonomics Society.
28.
JamesL. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219–229.
29.
JamesL. R.DemareeR. G.WolfG. (1993). rwg: An assessment of within-group interrater agreement. Journal of Applied Psychology, 78, 306–309.
30.
JonesD. G.EndsleyM. R. (2004). Use of real-time probes for measuring situation awareness. International Journal of Aviation Psychology, 14, 343–367.
31.
KeithN.FreseM. (2005). Self-regulation in error management training: Emotion control and metacognition as mediators of performance effects. Journal of Applied Psychology, 90, 677–691.
32.
KozlowskiS. W. J.KleinK. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco, CA: Jossey-Bass.
33.
KrugerJ.DunningD. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121–1134.
34.
LeeJ. D. (1999). Measuring driver adaptation to in-vehicle information systems: Disassociation of subjective and objective situation awareness measures. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 992–996). Santa Monica, CA: Human Factors and Ergonomics Society.
35.
LichaczF. M. J. (2008). Augmenting understanding of the relationship between situation awareness and confidence using calibration analysis. Ergonomics, 51, 1489–1502.
36.
LordR. G.DiefendorffJ. M.SchmidtA. M.HallR. J. (2010). Self-regulation at work. Annual Review of Psychology, 61, 543–568.
37.
MancusoV.HamiltonK.TeslerR.MohammedS.McNeeseM. (2013). An experimental evaluation of the effectiveness of endogenous and exogenous fantasy in computer-based simulation training. International Journal of Games and Computer-Mediated Simulations, 5, 50–65.
38.
MatthewsM. D.BealS. A. (2002). Assessing situation awareness in field training exercises (Research Report 1795). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
39.
McGuinnessB. (2004, June). Quantitative analysis of situational awareness (QUASA): Applying signal detection theory to true/false probes and self-ratings. Paper presented at the Command and Control Research and Technology Symposium, San Diego, CA.
40.
McGuinnessB.FoyL. (2000). A subjective measure of SA: The Crew Awareness Rating Scale. In KaberD. B.EndsleyM. R. (Eds.), Human performance, situation awareness and automation: User centered design for the new millennium (pp. 286–291). Atlanta, GA: SA Technologies.
41.
McNeeseM. D. (2000). Socio-cognitive factors in the acquisition and transfer of knowledge. Cognition, Technology & Work, 2, 164–177.
42.
McNeeseM. D.BainsP.BrewerI.BrownC.ConnorsE. S.JeffersonT.TerrellI. (2005). The NeoCITIES simulation: Understanding the design and experimental methodology used to develop a team emergency management simulation. In Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting (pp. 591–594). Santa Monica, CA: Human Factors and Ergonomics Society.
43.
MinotraD.BurnsC. M. (2015). Finding common ground: Situation awareness and cognitive work analysis. Journal of Cognitive Engineering and Decision Making, 9(1), 87-89.
44.
NelsonT. O.NarensL. (1994). Why investigate metacognition? In MetcalfeJ.ShimamuraA. P. (Eds.), Metacognition: Knowing about knowing (pp. 1–25). Cambridge, MA: MIT Press.
45.
NofiA. A. (2000). Defining and measuring shared situation awareness (No. CRM D0002895.A1/Final). Alexandria, VA: Center for Naval Analyses.
46.
PowersW. T. (1973). Behavior: The control of perception. New York, NY: Hawthorne.
47.
RousseauR.TremblayS.BanburyS.BretonR.GuitouniA. (2010). The role of metacognition in the relationship between objective and subjective measures of situation awareness. Theoretical Issues in Ergonomics Science, 11, 119–130.
48.
SalmonP. M.StantonN. A.WalkerG. H.GreenD. (2006). Situation awareness measurement: A review of applicability for C4i environments. Applied Ergonomics, 37, 225–238.
49.
SanerL. D.BolstadC. A.GonzalezC.CuevasH. M. (2009). Measuring and predicting shared situation awareness in teams. Journal of Cognitive Engineering and Decision Making, 3, 280–308.
50.
SchippersM. C.HomanA. C.van KnippenbergD. (2013). To reflect or not to reflect: Prior team performance as a boundary condition of the effects of reflexivity on learning and final team performance. Journal of Organizational Behavior, 34, 6–23.
51.
SchmidtA. M.DeShonR. P. (2009). Prior performance and goal progress as moderators of the relationship between self-efficacy and performance. Human Performance, 22, 191–203.
52.
SitzmannT.YeoG. (2013). A meta-analytic investigation of the within-person self-efficacy domain: Is self-efficacy a product of past performance or a driver of future performance?Personnel Psychology, 66, 531–568.
53.
StajkovicA. D.LeeD.NybergA. J. (2009). Collective efficacy, group potency and group performance: Meta-analyses of their relationships and test of a mediation model. Journal of Applied Psychology, 94, 814–828.
54.
SulistyawatiK.WickensC. D.ChuiY. P. (2009). Exploring the concept of team situation awareness in a simulated air combat environment. Journal of Cognitive Engineering and Decision Making, 3, 309–330.
55.
SundstromE. (1999). The challenges of supporting work team effectiveness. In SundstromE. & Associates (Eds.), Supporting work team effectiveness (pp. 3–23). San Francisco, CA: Jossey-Bass.
56.
SundstromE.de MeuseK. P.FutrellD. (1990). Work teams: Applications and effectiveness. American Psychologist, 45, 120–133.
57.
TaylorR. M. (1990). Situational Awareness Rating Technique (SART): The development of a tool for aircrew systems design (AGARD-CP-478). Neuilly Sur Seine, France: NATO-AGARD.
58.
UitdewilligenS.WallerM. J.ZijlstraF. R. H. (2010). Team cognition and adaptability in dynamic settings: A review of pertinent work. In HodgkinsonG. P.FordJ. K. (Eds.), International review of industrial and organizational psychology (Vol. 25, pp. 293–353). Hoboken, NJ: Wiley-Blackwell.
59.
WestM. A. (2000). Reflexivity, revolution and innovation in work teams. In BeyerleinM. M.JohnsonD. A.BeyerleinS. T. (Eds.), Product development teams (Vol. 5, pp. 1–29). Stamford, CT: JAI Press.
60.
WildmanJ. L.ThayerA. L.PavlasD.SalasE.StewartJ. E.HowseW. R. (2011). Team knowledge research: Emerging trends and critical needs. Human Factors, 54, 84–111.
