Information Order and Outcome Framing: An Assessment of Judgment Bias in a Naturalistic Decision-Making Context

Abstract

Findings that decision makers can come to different conclusions depending on the order in which they receive information have been termed the "information order bias." When trained, experienced individuals exhibit similar behaviors; however, it has been argued that this result is not a bias, but rather, a patternmatching process. This study provides a critical examination of this claim. It also assesses both experts' susceptibility to an outcome framing bias and the effects of varying task loads on judgment. Using a simulation of state-of-the-art ship defensive systems operated by experienced, active-duty U.S. Navy officers, we found no evidence of a framing bias, while task load had a minor, but systematic effect. The order in which information was received had a significant impact, with the effect being consistent with a judgment bias. Nonetheless, we note that patternmatching processes, similar to those that produce inferential and reconstructive effects on memory, could also explain our results. Actual or potential applications of this research include decision support system interfaces or training programs that might be developed to reduce judgment bias.

Get full access to this article

View all access options for this article.

References

Adelman, L. , Bresnick, T. , Black, P. K. , Marvin, F. F. , & Sak, S. G. (1996). Research with Patriot Air Defense officers: Examining information order effects. Human Factors, 38, 250–261.

Adelman, L. , Tolcott, M. A. , & Bresnick, T. A. (1993). Examining the effect of information order on expert judgment. Organizational Behavior and Human Decision Processes, 56, 348–369.

Anderson, M. R. (1991). The adaptive nature of human categorization. Psychological Review, 98, 409–429.

Ashton, A. H. , & Ashton, R. H. (1988). Sequential belief revision in auditing. The Accounting Review, 63, 623–641.

Ashton, R. H. , & Ashton, A. H. (1990). Evidence responsiveness in professional judgment: Effects of positive versus negative evidence and presentation mode. Organizational Behavior and Human Decision Processes, 46, 1–19.

Bartlett, F. C. (1932). Remembering: A study in experimental and social psychology New York: Cambridge University Press.

Cannon-Bowers, J. A. , Salas, E. , & Pruitt, J. S. (1996). Establishing the boundaries of a paradigm for decision-making research. Human Factors, 38, 193–205.

Entin, E. E. (1992). Inertia and adaptation in the sequential processing of information. In Proceedings of the psychology in the Department of Defense 13th symposium, 193--197. Colorado Springs, CO: Department of Behavioral Sciences and Leadership, U.S. Air Force Academy.

Entin, E. E. , Serfaty, D. , & Forester, J. (1989). Sequential processing of information from multiple sources. In Proceedings of the system, man, and cybernetics society, (pp. 91–100). New York: IEEE Press.

10.

Hogarth, R. M. (1987). Judgment and choice (2nd ed.). New York: Wiley-Interscience.

11.

Hogarth, R. M. , & Einhorn, H. J. (1992). Order effects in belief updating: The belief-adjustment model. Cognitive Psychology, 24, 1–55.

12.

Kahneman, D. , Slovic, P. , & Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases New York: Cambridge University Press.

13.

Klein, G. A. , Orasanu, J. , Calderwood, R. , & Zsambok, C. E. (1993). Decision making in action: Models and methods. Norwood, NJ: Ablex.

14.

Knapp, B. G. , & Tolbert, C. A. (1985). Insights on information absorption and transmission rates in C2I settings (ARI Research Note 85-93). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

15.

Nisbett, R. E. , & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment Englewood Cliffs, NJ: Prentice-Hall.

16.

Rosch, E. (1973). On the internal structure of perceptual and semantic categories. In T. E. Moore (Ed.), Cognitive development and the acquisition of language (pp. 111–144). New York: Academic Press.

17.

Salas, E. & Cannon-Bowers, J. A. (Eds.) (1996). Decision making in complex environments [Special issue]. Human Factors, 38(2).

18.

Schank, R. C. , & Abelson, R. (1977). Scripts, plans, goals, and understanding Mahwah, NJ: Erlbaum.

19.

Sheridan, T. (1981). Understanding human error and aiding human diagnostic behavior in nuclear power plants. In J. Rasmussen & W. Rouse (Eds.), Human detection and diagnosis of system failures (pp. 19–35). New York: Plenum Press.

20.

Tolcott, M. A. , Marvin, F. F. , & Lehner, P. E. (1989). Expert decision making In evolving situations. IEEE Transactions on Systems, Man, and Cybernetics, 19, 606–615.

21.

Tversky, A. , & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453–458.

22.

Wierwille, W. W. , & Casali, J. G. (1983). A validated rating scale for global mental workload measurement applications. In Proceedings of the Human Factors Society 27th Annual Meeting, 129--133. Santa Monica: Human Factors Society.

23.

Winer, B. F. (1962). Statistical principles In experimental design New York: McGraw-Hill.

24.

Zsambok, C. E. , & Klein, G. (1997). Naturalistic decision making Mahwah, NJ: Erlbaum.