Efforts to improve decision making must appeal to some source of warrant - that is, specific criteria or models for guiding and evaluating decision-making performance. We examine and compare the warrants for two approaches to decision aids, decision training, and consulting: analytically based prescription, which obtains warrant from formal models, and empirically based prescription, which obtains warrant from descriptive models of successful performance. We argue that empirically based warrants can provide a meaningful and valid basis for prescriptive intervention without committing the naturalistic fallacy (i.e., confusing what is with what ought to be) and without the use of formal deduction from first principles. We describe points of divergence as well as convergence in the types of warrant appealed to by naturalistic decision making and decision analysis, letting each approach shed light on the other, and explore the application of empirically based prescription to cognitive engineering. Actual or potential applications of this research include the development of training programs to improve various aspects of naturalistic decision making.
Get full access to this article
View all access options for this article.
References
1.
Baker, L. (1985). How do we know when we don't understand? Standards for evaluating text comprehension. In D. L. Forrest-Pressley, & T. G. Waller (Eds.), Metacognition, cognition, and human performance (Vol. 1, pp. 155–205). Orlando, FL: Academic.
2.
Baron, J., & Brown, R. V. (1991). Teaching decision making to adolescents.Mahwah, NJ: Erlbaum.
3.
Bazerman, M. H. (1994). Judgment in management decision making (2nd ed.). New York: Wiley.
4.
Bell, D. E., Raiffa, H., & Tversky, A. (1988). Descriptive, normative, and prescriptive interactions in decision making. In D. E. Bell, H. Raiffa, & A. Tversky (Eds.), Decision making: Descriptive, normative, and prescriptive interactions (pp. 9–30). Cambridge, UK: Cambridge University Press.
5.
Brown, R. V., Kahr, A. S., & Peterson, C. (1974). Decision analysis for the manager.New York: Holt, Rinehart and Winston.
6.
Brown, R. V., & Ulvila, J. W. (1976). Selecting analysis approaches for decision situations: A matching of taxonomies.McLean, VA: Decisions and Designs.
7.
Bruner, J. (1996). Frames for thinking. In D. R. Olson, & N. Torrance (Eds.), Modes of thought: Explorations in culture and cognition (pp. 93–105). Cambridge, UK: Cambridge University Press.
8.
Calderwood, R., Klein, G., & Crandall, B. W. (1988). Time pressure, skill, and move quality in chess. American Journal of Psychology, 101, 481–491.
9.
Chase, W., & Simon, H. (1973). The mind's eye in chess. In W. Chase (Ed.), Visual information processing (pp. 215–281). New York: Academic.
10.
Cherniak, C. (1986). Minimal rationality.Cambridge, MA: MIT Press.
11.
Chi, M., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. S. Sternberg (Ed.), Advances in the psychology of human intelligence (Vol. 1, pp. 7–75). Hillsdale, NJ: Erlbaum.
12.
Cohen, L. J. (1981). Can human irrationality be experimentally demonstrated?Brain and Behavioral Sciences, 4, 510–517.
13.
Cohen, M. S. (1993). The naturalistic basis of decision biases. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 51–102). Norwood, NJ: Ablex.
14.
Cohen, M. S. (2002). Training critical thinking to enhance battlefield initiative. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 2002–2006). Santa Monica, CA: Human Factors and Ergonomics Society.
15.
Cohen, M. S. (2004). Leadership as the orchestration and improvisation of dialogue: Cognitive and communicative skills in conversations among leaders and subordinates. In D. V. Day, S. J. Zaccaro, & S. M. Halpin (Eds.), Leader development for transforming organizations: Growing leaders for tomorrow (pp. 177-208). Mahwah, NJ: Erlbaum.
16.
Cohen, M. S., Adelman, L., & Thompson, B. T. (2000). Experimental investigation of uncertainty, stakes, and time in pilot decision making.Arlington, VA: Cognitive Technologies.
17.
Cohen, M. S., & Freeman, J. T. (1997). Understanding and enhancing critical thinking in recognition-based decision-making. In R. Flin, E. Salas, M. Strub, & L. Martin (Eds.), Decision making under stress: Emerging themes and applications (pp. 161–169). Aldershot, UK: Ashgate.
18.
Cohen, M. S., Freeman, J. T., & Thompson, B. (1997). Training the naturalistic decision maker. In C. E. Zsambok, & G. Klein (Eds.), Naturalistic decision making (pp. 257–268). Mahwah, NJ: Erlbaum.
19.
Cohen, M. S., Freeman, J. T., & Thompson, B. (1998). Critical thinking skills in tactical decision making: A model and a training strategy. In J. A. Cannon-Bowers, & E. Salas (Eds.), Decision making under stress: Implications for training and simulation (pp. 155–189). Washington, DC: American Psychological Association.
20.
Cohen, M. S., Freeman, J. T., & Wolf, S. (1996). Meta-recognition in time stressed decision making: Recognizing, critiquing, and correcting. Human Factors, 38, 206–219.
21.
Cohen, M. S., Salas, E., & Thompson B. B. (2001). Critical thinking: A normative, cognitive, and pragmatic framework.Arlington, VA: Cognitive Technologies.
22.
Cohen, M. S., & Thompson, B. B. (2001). Training teams to take initiative: Critical thinking in novel situations. In E. Salas (Ed.), Advances in human performance and cognitive engineering research (Vol. 1, pp. 251–291). Greenwich, CT: JAI.
23.
Cohen, M. S., Thompson, B. B., Adelman, L., Bresnick, T. A., Shastri, L., & Riedel, S. (2000a). Training critical thinking for the battlefield: Volume II. Training system and evaluation.Arlington, VA: Cognitive Technologies.
24.
Cohen, M. S., Thompson, B. B., Adelman, L., Bresnick, T. A., Shastri, L., & Riedel, S. (2000b). Training critical thinking for the battlefield: Volume III. Modeling and simulation of battlefield critical thinking.Arlington, VA: Cognitive Technologies.
25.
Crandall, B., & Getchell-Reiter, K. (1993). Critical decision method: A technique for eliciting concrete assessment indicators from the intuition of NICU nurses. Advances in Nursing Science, 16(1), 42–51.
26.
Edwards, W. (1983). Simplicity in decision analysis: Two case studies.Cambridge, MA: Harvard College.
27.
Einhorn, H. J., & Hogarth, R. M. (1985). Ambiguity and uncertainty in probabilistic inference. Psychological Review, 92, 433–461.
28.
Ellsberg, D. (1988). Risk, ambiguity, and the Savage axioms. In P. Gardenfors & N.-E. Sahlin (Eds.), Decision, probability, and utility: Selected reasoning (pp. 245–269). Cambridge, UK: Cambridge University Press.
29.
Ericsson, K. A. (2005). Superior decision making as an integral quality of expert performance: Insights into the mediating mechanisms and their acquisition through deliberate practice. In B. Brehmer, R. Lipshitz, & H. Montgomery (Eds.), How professionals make decisions (pp. 135–167). Mahwah, NJ: Erlbaum.
30.
Ericsson, K. A., & Smith, J. (Eds.). (1991). Toward a general theory of expertise: Prospects and limits.Cambridge, UK: Cambridge University Press.
31.
Fischhoff, B. (2000). Informed consent in eliciting environmental values. Environmental Science and Technology, 38, 1439–1444.
32.
Forrest-Pressley, D. L., MacKinnon, G. E., & Waller, T. G. (1985). Metacognition, cognition, and human performance.New York: Academic.
33.
Freeman, J. T., Thompson, B. T., & Cohen, M. S. (1999). Modeling and diagnosing domain knowledge using latent semantic indexing. In Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (pp. 233–236). Santa Monica, CA: Human Factors and Ergonomics Society.
34.
Gavelek, J., & Raphael, T. E. (1985). Metacognition, instruction, and the role of questioning activities. In D. L. Forrest-Pressley, G. E. MacKinnon, & T. G. Waller (Eds.), Metacognition, cognition, and human performance (Vol. 2, pp. 103–136). New York: Academic.
35.
Gibbard, A., & Harper, W. L. (1988). Counterfactuals and two kinds of expected utility. In P. Gardenfors & N.-E. Sahlin (Eds.), Decision, probability, and utility: Selected reasoning (pp. 341–376). Cambridge, UK: Cambridge University Press.
36.
Gigerenzer, G. (2000). Adaptive thinking: Rationality in the real world.Oxford, UK: Oxford University Press.
37.
Gilovich, T., Griffin, D., & Kahneman, D. (Eds.). (2002). Heuristics and biases: The psychology of intuitive judgment.Cambridge, UK: Cambridge University Press.
38.
Goodman, N. (1965). Fact, fiction, and forecast.Indianapolis, IN: Bobbs-Merrill.
39.
Gordon, S. E., & Gill, R. T. (1997). Cognitive task analysis. In C. E. Zsambok, & G. Klein (Eds.), Naturalistic decision making (pp. 131–140). Mahwah, NJ: Erlbaum.
40.
Hammond, J. S., Keeney, R. L., & Raiffa, H. (1998). Smart choices: A practical guide to making better decisions.Cambridge, MA: Harvard University Press.
41.
Hammond, K. R. (1996). Human judgment and social policy: Irreducible uncertainty, inevitable error, unavoidable injustice.Oxford, UK: Oxford University Press.
42.
Harman, G. (1986). Change in view: Principles of reasoning.Cambridge, MA: MIT Press.
43.
Hastie, R. (1993). Inside the juror.Cambridge, UK: Cambridge University Press.
44.
Hoffman, R. R., Crandall, B., & Shadbolt, N. (1998). Use of critical decision method to elicit expert knowledge: A case study in the methodology of expert task analysis. Human Factors, 40, 254–276.
45.
Kaempf, G. F., Klein, G., Thordsen, M. L., & Wolf, S. (1996). decision making in complex command-and-control environments. Human Factors, 38, 206–219.
46.
Kahneman, D., Slovic, P. A., & Tversky, A. (Eds.). (1982). Judgment under uncertainty: Heuristics and biases.New York: Cambridge University Press.
47.
Kahneman, D., & Tversky, A. (Eds.). (2000). Choice, values, and frames.Cambridge, UK: Cambridge University Press.
48.
Kaplan, M. (1996). Decision theory as philosophy.Cambridge, UK: Cambridge University Press.
49.
Keeney, R. L. (1999, August). From "decisions with multiple objectives" to "smart choices". Paper presented at the 17th Conference on Subjective Probability, Utility, and Decision Making, Mannheim, Germany.
50.
Keeney, R. L., & Raiffa, H. (1976). Decisions with multiple objectives: Preferences and value trade-offs.New York: Wiley.
51.
Klein, G. (1993). Recognition-primed decision (RPD) model of rapid decision making. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138–147). Norwood, NJ: Ablex.
52.
Klein, G. (1997). Developing expertise in decision making. Thinking and Reasoning, 34, 337–352.
53.
Klein, G. (1998). Sources of power: How people make decisions.Cambridge, MA: MIT.
54.
Klein, G., Kaempf, G. L., Thordsen, M., & Miller, T. (1996). The uses of decision requirement. International Journal of Human-Computer Studies, 46, 1–15.
55.
Klein, G., Orasanu, J., Calderwood, R., & Zsambok, C. E. (Eds.). (1993). Decision making in action: Models and methods.Norwood, NJ: Ablex.
56.
Klein, G., & Wolf, S. (1995). Decision-centered training. In Proceedings of the Human Factors and Ergonomics Society 39th Annual Meeting (pp. 1249–1252). Santa Monica, CA: Human Factors and Ergonomics Society.
57.
Kleinmuntz, D., Schoemaker, P. J. H., & Kunreuther, H. C. (1993). Decision sciences.Cambridge, UK: Cambridge University Press.
58.
Kuhn, D., Amsel, E., & O'Loughlin, M. (1988). The development of scientific thinking skills.San Diego: Academic.
59.
LeBoeuf, R., & Shafir, E. (2001). Problems and methods in naturalistic decision making research. Journal of Behavioral Decision Making, 14, 374–375.
60.
Lipshitz, R. (1994). Decision making in three modes. Journal for the Theory of Social Behavior, 24, 47–66.
61.
Lipshitz, R., & Adar-Pras, A. (2004). Recognition-primed decisions in the laboratory. In H. Montgomery, R. Lipshitz, & B. Brehmer, (Eds.), How professionals make decisions (pp. 91–106). Hillsdale, NJ: Erlbaum.
62.
Lipshitz, R., Klein, G., Orasanu, J., & Salas, E. (2001). Focus article: Taking stock of naturalistic decision making. Journal of Behavioral Decision Making, 14, 331–352.
63.
MacCrimmon, K. R. (1968). Descriptive and normative implications of the decision theory postulates. In K. Borch, & J. Mossin (Eds.), Risk and uncertainty (pp. 3–23). New York: St Martin's.
64.
March, J. G., & Heath, C. (1994). A primer on decision making.New York: Free Press.
65.
McClennen, E. F. (1990). Rationality and dynamic choice: Foundational explorations.Cambridge, UK: Cambridge University Press.
66.
Means, B., Salas, E., Crandall, B., & Jacobs, T. O. (1993). Training decision makers for the real world. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 306–326). Norwood, NJ: Ablex.
67.
Montgomery, H. (1993). The search for a dominance structure in decision making: Examining the evidence. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 182–187). Norwood, NJ: Ablex.
68.
Neisser, U. (1998). Stories, selves, and schemata: A review of ecological findings. In S. Gathercole, & C. Cornoldi (Eds.), Theories of memory (Vol. 2, pp. 171–186). Hove, UK: Psychology Press.
69.
Nisbett, R. E. (Ed.). (1993). Rules for reasoning.Hillsdale, NJ: Erlbaum.
70.
Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment.Englewood Cliffs, NJ: Prentice-Hall.
71.
Nozick, R. (1993). The nature of rationality.Princeton, NJ: Princeton University Press.
72.
Nutt, P. C. (1999). Surprising but true: Half the decisions in organizations fail. Academy of Management Executive, 13, 75–90.
73.
Patel, V. L., & Groen, G. J. (1991). The general and specific nature of medical expertise: A critical look. In K. A. Ericsson, & J. Smith (Eds.), Towards a general theory of expertise (pp. 93–125). Cambridge, UK: Cambridge University Press.
74.
Pearl, J. (2000). Causality: models, reasoning, and inference.Cambridge, UK: Cambridge University Press.
75.
Pennington, N., & Hastie, R. (1993). A theory of explanation-based decision making. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 188–201). Norwood, NJ: Ablex.
76.
Pitz, G. F. (1992). Risk taking, design, and training. In J. F. Yates (Ed.), Risk-taking behavior (pp. 283–329). New York: Wiley.
77.
Plantinga, A. (1993). Warrant: The current debate.Oxford, UK: Oxford University Press.
78.
Pliske, R., McCloskey, R., & Klein, G. (2001). Facilitating learning from experience: An innovative approach to decision skills training. In E. Salas, & G. Klein (Eds.), Linking expertise and naturalistic decision making (pp. 37–53). Mahwah, NJ: Erlbaum.
79.
Popper, K. (1995). Conjectures and refutations.New York: HarperCollins. (Original work published 1963)
80.
Quine, W. V. O., & Ullian, J. S. (1970). The web of belief.New York: Random House.
81.
Raiffa, H. (1968). Decision analysis: Introductory lectures on choices under uncertainty.Reading, MA: Addison Wesley.
82.
Raiffa, H. (1994). The prescriptive orientation of decision making: A synthesis of decision analysis, behavioral decision making, and game theory. In S. Rios (Ed.), Decision theory and decision analysis: Trends and challenges (pp. 3-13). Dordrecht, Netherlands: Kluwer.
83.
Ramsey, F. (1931). Truth and probability. In F. Ramsey (Ed.), The foundations of mathematics and other logical essays (pp. 156-198). New York: Harcourt, Brace.
84.
Rescher, N. (1977). Methodological pragmatism: A systems-theoretic approach to the theory of knowledge.New York: New York University Press.
85.
Rubin, H. (1988). Discussion of A. P. Dempster, probability, evidence, and judgment. In D. E. Bell, H. Raiffa, & A. Tversky (Eds.), Decision making: Descriptive, normative, and prescriptive interactions (pp. 292-293). Cambridge, UK: Cambridge University Press.
86.
Russo, E. J., & Schoemaker, P. J. H. (1987). Decision traps: Ten barriers to brilliant decision making and how to overcome them.New York: Doubleday.
87.
Savage, L. J. (1954). The foundations of statistics.New York: Wiley.
88.
Schank, R. C. (1990). Tell me a story: Narrative and intelligence.Evanston, IL: Northwestern University Press.
89.
Schick, F. (1997). Making choices: A recasting of decision theory.Cambridge, UK: Cambridge University Press.
90.
Schum, D. A. (1994). The evidential foundations of probabilistic reasoning.New York: Wiley.
91.
Sellars, W. (2000). Epistemic principles. In E. Sosa, & J. Kim (Eds.), Epistemology: An anthology (pp. 125-133). Malden, MA: Blackwell. (Original work published 1975)
92.
Shafer, G., & Tversky, A. (1988). Languages and designs for probability judgment. In D. E. Bell, H. Raiffa, & A. Tversky (Eds.), Decision making: Descriptive, normative, and prescriptive interactions (pp. 237-265). Cambridge, UK: Cambridge University Press.
93.
Shafir, E., Simonson, I., & Tversky, A. (1993). Reason-based choice. Cognition, 49, 11–36.
94.
Shanteau, J. (1992). The psychology of experts: An alternative view. In G. Wright, & F. Bolger (Eds.), Expertise and decision support (pp. 252-266). New York: Plenum.
95.
Simon, H. (1988). Rationality as process and as product of thought. In D. E. Bell, H. Raiffa, & A. Tversky (Eds.), Decision making: Descriptive, normative, and prescriptive interactions (pp. 58-77). Cambridge, UK: Cambridge University Press.
96.
Slovic, P. (1982). Toward understanding and improving decisions. In M. D. Dunnette, & E. A. Fleishman (Eds.), Human capability assessment (pp. 157-183). Hillsdale, NJ: Erlbaum.
97.
Slovic, P. (2000). The construction of preference. In D. Kahneman, & A. Tversky (Eds.), Choice, values, and frames (pp. 489-502). Cambridge, UK: Cambridge University Press.
98.
Slovic, P., & Tversky, A. (1974). Who accepts Savage's axioms?Behavioral Science, 19, 368–373.
99.
Stanovich, K. E. (1999). Who is rational? Studies of individual differences in reasoning.Hillsdale, NJ: Erlbaum.
100.
Stich, S. (1991). The fragmentation of reason.Cambridge, MA: MIT Press.
101.
Taynor, J., Crandall, B., & Wiggins, S. (1987). The reliability of the critical decision method (Contract MDA-903-C-0170). Alexandria, VA: U.S. Army Research Institute Field Unit.
102.
Thompson, B. B., & Cohen, M. S. (1999). Naturalistic decision making and models of computational intelligence. In A. Jagota, T. Plate, L. Shastri, & R. Sun (Collective article Eds.), Connectionist symbol processing: Dead or alive?Neural Computing Surveys, 226–28.
103.
von Winterfeldt, D. (1999). On the relevance of behavioral decision research for decision analysis. In J. Shanteau, B. Mellers, & D. Schum (Eds.), Decision science and technology: Reflections on the contributions of Ward Edwards (pp. 133-154). Dordrecht, Netherlands: Kluwer Academic.
104.
von Winterfeldt, D., & Edwards, W. (1986). Decision analysis and behavioral research.Cambridge, UK: Cambridge University Press.
105.
Yates, J. F. (2001). "Outside" impressions of naturalistic decision making. In E. Salas, & G. Klein (Eds.), Linking expertise and naturalistic decision making (pp. 9–33). Mahwah, NJ: Erlbaum.
