The authors report two experiments studying the requirements for effective decision making in a complex environment. The focus lies on three components of individual cognitive readiness: situation awareness (SA), problem solving, and decision making. Participants performed a simulated society management task in which they could allocate resources to stabilize a national crisis involving multiple interrelated factors (political, economic, environmental, and social). A striking aspect of this simulation is that even though information about the causes and effects within the system is available, most individuals fail to bring the system to the targeted state because of unintended consequences of their decisions. The experiments test the impact of two cognitive support tools designed to improve anticipation of future outcomes. Results show that supporting short-term anticipation (with perfectly accurate projections) was insufficient to improve effectiveness, but supporting long-term anticipation (with approximate projections) successfully improved performance in this complex environment. We conclude with a review of requirements that training and technological support should address to augment individual cognitive readiness for operations in complex environments and propose an extension to SA theory by conceptualizing a Level 4 SA (long-term projection) that may be particularly important to overcome the “wall of complexity.”
AllenJ. G.CorpacP. S.FrisbieK. R. (2006, June). Integrated battle command program: Decision support tools for planning and conducting unified action campaigns in complex contingencies. Paper presented at the Command and Control Research and Technology Symposium, San Diego, CA.
2.
ArnoldV.CollierP. A.LeechS. A.SuttonS. G. (2004). Impact of intelligent decision aids on expert and novice decision-makers’ judgments. Accounting & Finance, 44, 1–26.
3.
BanachS. J. (2009, March/April). Educating by design: Preparing leaders for a complex world. Military Review, pp. 96–104.
4.
BorsD.StokesT. L. (1998). Raven’s advanced progressive matrices: Norms for first-year university students and the development of a short form. Education and Psychological Measurement, 58, 382–398.
5.
BusemeyerJ. R.PleskacT. J. (2009). Theoretical tools for understanding and aiding dynamic decision making. Journal of Mathematical Psychology, 53, 126–138.
6.
CarafanoJ. J.WeitzR. (2009). Complex systems analysis: A necessary tool for homeland security. Backgrounder, 2261, 1–7.
7.
ChandrasekaranB.GoldmanM. (2007). Exploring robustness of plans for simulation-based course of action planning. In Proceedings of the IEEE Symposium on Computational Intelligence in Multicriteria Decision Making (pp. 185–192). New York, NY: IEEE.
8.
ChenJ. Q.LeeS. M. (2003). An exploratory cognitive DSS for strategic decision making. Decision Support Systems, 36, 147–160.
9.
DörnerD. (1996). The logic of failure: Recognizing and avoiding error in complex situations. Reading, MA: Addison-Wesley.
10.
DuCharmeM. B.LafondD.RathbunB. (2011). A gaming approach to train systems thinking in the military. In Proceedings of the Eighth International Conference on Complex Systems (pp. 851–852). Quincy, MA: New England Complex Systems Institute.
11.
ElgF. (2005). Leveraging intelligence for high performance in complex dynamic systems requires balanced goals. Theoretical Issues in Ergonomics Science, 6, 63–72.
12.
EndsleyM. R. (1988). Situation awareness global assessment technique (SAGAT). In Proceedings of the National Aerospace and Electronics Conference (pp. 789–795). New York, NY: IEEE.
13.
EndsleyM. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37, 32–64.
14.
EndsleyM. R. (2001, November). Designing for situation awareness in complex systems. Paper presented at Second International Workshop on Symbiosis of Humans, Artifacts and Environment, Kyoto, Japan.
15.
EtterD. M.FosterR. E.SteeleT. P. (2000). Cognitive readiness and advanced distributed learning. Crosstalk: The Journal of Defense Software Engineering, 13, 5–6.
16.
FletcherJ. D. (2004). Cognitive readiness: Preparing for the unexpected. Alexandria, VA: Institute for Defense Analyses.
17.
FunkeJ. (1991). Solving complex problems: Exploration and control of complex systems. In SternbergR. J.FrenschP. A. (Eds.), Complex problem solving: Principles and mechanisms (pp. 185–222). Hillsdale, NJ: Lawrence Erlbaum.
18.
FunkeJ. (1993). Microworlds based on linear equation systems: A new approach to complex problem solving and experimental results. In StrubeG.WenderK.-F. (Eds.), The cognitive psychology of knowledge (pp. 313–330). Amsterdam, Netherlands: Elsevier.
19.
GonzalezC. (2005). Decision support for real-time, dynamic decision-making tasks. Organizational Behavior and Human Decision Processes, 96, 142–154.
20.
GonzalezC.ThomasR. P.VanyukovP. (2005). Impact of individual differences and cognitive abilities on dynamic decision making. Intelligence, 33, 169–186.
21.
GoodeN.BeckmannJ. F. (2010). You need to know: There is a causal relationship between structural knowledge and control performance in complex problem solving tasks. Intelligence, 38, 345–352.
22.
Grossman-VermaasR. (2004). Future perfect: Effects based operations, complexity and the human environment (DOR Joint Research Note RN 2004/01). Ottawa, Canada: Department of National Defence Canada.
23.
GureckisT.LoveB. (2009). Short-term gains, long-term pains: How cues about state aid learning in dynamic environments. Cognition, 113, 293–313.
24.
HalfordG. S.BakerR.McCreddenJ. E.BainJ. D. (2005). How many variables can humans process?Psychological Science, 16, 70–76.
25.
HornB. (2003, Autumn). Complexity squared: Operating in the future battlespace. Canadian Military Journal, pp. 7–15.
26.
JägerA. O. (1982). Mehrmodale klassifikation von intelligenzleistungen: Experimentell kontrollierte weiterentwicklung eines deskriptiven intelligenzstrukturmodells [Multimodal classification of intelligence tests: Experimentally controlled development of a descriptive model of intelligence structure]. Diagnostica, 28, 195–225.
27.
JarmaszJ. (2007). Accelerated training with microworlds for command decision making. In Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting (pp. 1578–1582). Santa Monica, CA: Human Factors and Ergonomics Society.
28.
KarakulM.Qudrat-UllahH. (2008). How to improve dynamic decision-making? Practice and promise. In Qudrat-UllahH.SpectorJ. M.DavidsenP. I. (Eds.), Complex decision making: Theory and practice (pp. 3–24). Berlin, Germany: Springer.
29.
KleinG.MoonB.HoffmanR. F. (2006). Making sense of sensemaking I: Alternative perspectives. IEEE Intelligent Systems, 21, 70–73.
30.
KleinG.PfaffM.DruryJ. L. (2009, March). Supporting a robust decision space. Paper presented at the AAAI Spring Symposium on Technosocial Predictive Analytics, Stanford, CA.
31.
KurtzC. F.SnowdenD. J. (2003). The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal, 42, 462–483.
32.
LafondD.DuCharmeM. B. (2011). Complex decision making experimental platform (CODEM): A counter-insurgency scenario. In Proceedings of the IEEE Symposium on Computational Intelligence for Security and Defence Applications (pp. 72–79). New York, NY: IEEE.
33.
LafondD.TremblayS.RiouxF.DuCharmeM. B. (2011). Training systems thinking skills using microworlds: Integration of metrics for intelligent tutoring. In Proceedings of the 2011 United Kingdom System Society Conference (pp. P1). Oxford, UK: UK Systems Society.
34.
LafondD.VachonF.RousseauR.TremblayS. (2010). A cognitive and holistic approach to developing metrics for decision support in command and control. In KaberD.BoyG. (Eds.), Advances in cognitive ergonomics (pp. 65–75). Boca Raton, FL: CRC Press.
35.
LangleyP. A.MorecroftJ. D. W. (2004). Performance and learning in a simulation of oil industry dynamics. European Journal of Operational Research, 155, 715–732.
36.
LangtonJ.DasS. (2007). A framework for building and reasoning with adaptive and interoperable PMESII models (Air Force Research Laboratory Tech. Rep. AFRL-RI-RS-TR-2007-241). Cambridge, MA: Charles River Analytics.
37.
LerchF. J.HarterD. E. (2001). Cognitive support for real-time dynamic decision making. Information Systems Research, 12, 63–82.
38.
LizotteM.BernierF.MokhtariM.BoivinE.DuCharmeM.PoussartP. (2008, December). Image: Simulation for understanding complex situations and increasing Future Force agility. Paper presented at the 26th Army Science Conference, Orlando, FL.
39.
MandrikC. A.BaoY. (2005). Exploring the concept and measurement of general risk aversion. Advances in Consumer Research, 32, 531–539.
40.
MillerA. (1982). Tunnel vision in environmental management. Environmentalist, 2, 223–231.
41.
MorrisonJ. E.FletcherJ. D. (2002). Cognitive readiness (IDA Paper P-3735, Contract DASW01 98C0067). Alexandria, VA: Institute for Defense Analyses.
42.
NewellA.SimonH. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.
43.
NortonR. W. (1975). Measurement of ambiguity tolerance. Journal of Personality Assessment, 39, 607–619.
44.
PoultonE. (1989). Bias in quantifying judgment. Hillsdale, NJ: Lawrence Erlbaum.
45.
PreußlerW. (1998). Strukturwissen als voraussetzung für die steuerung komplexer dynamischer systeme [Structural knowledge as precondition for the control of complex systems]. Zeitschrift für Experimentelle Psychologie, 45, 218–240.
46.
Putz-OsterlohW. (1993). Strategies for knowledge acquisition and transfer of knowledge in dynamic tasks. In StrubeG.WenderK. F. (Eds.), The cognitive psychology of knowledge (pp. 331–350). Amsterdam, Netherlands: Elsevier.
47.
RigasG.BrehmerB. (1999). Mental processes in intelligence tests and dynamics decision making tasks. London, UK: Lawrence Erlbaum.
48.
RousseauC. (2003, Summer). Complexity and the limits of modern battlespace visualization. Canadian Military Journal, pp. 35–44.
49.
SchoppekW. (2002). Examples, rules, and strategies in the control of dynamic systems. Cognitive Science Quarterly, 2, 63–92.
50.
SengeP. M. (1990). The fifth discipline: The art and practice of the learning organization (1st ed.). New York, NY: Doubleday/Currency.
51.
SmithE. A. (2006). Complexity, networking, and effects based approaches to operations. Washington, DC: CCRP.
52.
SorrellsW. T.DowningG. R.BlakesleyP. J.PendallD. W.WalkJ. K.WallworkR. D. (2005). Systemic operational design: An introduction. Fort Leavenworth, KS: School of Advanced Military Studies, United States Army Command and General Staff College.
53.
StermanJ. D. (2006). Learning from evidence in a complex world. American Journal of Public Health, 96, 505–514.
54.
ThunholmP. (2009). Military leaders and followers: Do they have different decision styles?Scandinavian Journal of Psychology, 50, 317–324.
55.
VesterF. (2007). The art of interconnected thinking: Ideas and tools for tackling with complexity. Munich: MCB-Verlag.
56.
WickensC. D. (1992). Engineering psychology and human performance (2nd ed.). New York, NY: HarperCollins.
57.
WittmannW. W.HattrupK. (2004). The relationship between performance in dynamic systems and intelligence. Systems Research and Behavioral Science, 21, 393–409.
58.
WoodsD. D.HollnagelE. (2006). Joint cognitive systems: Patterns in cognitive systems engineering. Boca Raton, FL: CRC Press.
59.
YasarcanH. (2009). Improving understanding, learning, and performances of novices in dynamic managerial simulation games: A gradual-increase-in-complexity approach. Complexity, 15(4), 31–42.
