People's reasoning with uncertainty information is often flawed. Visual representations can help, but little is known about what is the best way to present such information. Two studies investigated the effect of visualizations on the understanding and use of wind speed forecast uncertainty. Participants varied in expertise from novices in weather forecasting (Experiment 1) to professional forecasters (Experiment 2). The authors investigated three visualizations: (a) a chart showing the amount of uncertainty, (b) a chart showing the worst-case scenario, and (c) a box plot of likely wind speeds. Participants were asked to determine the relative uncertainty in the forecast, predict wind speed, and decide whether to post a high-wind warning advisory. The results for novices and professional forecasters were similar. The uncertainty chart enhanced awareness of the degree of uncertainty associated with the forecast, box plots improved reading accuracy, and presenting a visualization of the worst-case scenario introduced bias in the deterministic wind speed forecast. An interactive display (e.g., a combination of an uncertainty chart with a box plot display) may be optimal to convey uncertainty information.
Get full access to this article
View all access options for this article.
References
1.
BrownB. G.MurphyA. H. (1987). Quantification of uncertainty in fire-weather forecasts: Some results of operational and experimental forecasting programs. Weather and Forecasting, 2, 190–205.
2.
CamererC.HoT. (1994). Violations of the betweenness axiom and nonlinearity in probability. Journal of Risk and Uncertainty, 8, 167–196.
3.
ChapmanG. B.JohnsonE. J. (2002). Incorporating the irrelevant: Anchors in judgments of belief and value. In GilovichT.GriffinD.KahnemanD. (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 120–138). Cambridge, UK: Cambridge University Press.
4.
ColeW. G. (1989). Understanding Bayesian reasoning via graphical displays. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Wings for the Mind, 20, 381–386.
5.
deEliaR.LapriseR. (2005). Diversity in interpretations of probability: Implications for weather forecasting. American Meteorological Society, 133, 1129–1143.
6.
EndsleyM.BoltéB.JonesD. (2003). Designing for situation awareness: An approach to user-centered design. London: Taylor & Francis.
7.
GigerenzerG.HertwigR.van den BroekE.FasoloF.KatsikopoulosK. V. (2005). “A 30% chance of rain tomorrow”: How does the public understand probabilistic weather forecasts? Risk Analysis, 25, 623–630.
8.
GneitingT.RafteryA. E. (2005). Weather forecasting with ensemble methods. Science, 310, 248–249.
9.
GonzalezR.WuG. (1999). On the form of the probability weighting function. Cognitive Psychology, 38, 129–166.
10.
HeathC.LuffP. (2000). Technology in action. Cambridge, UK: Cambridge University Press.
11.
HoffmanJ. R.WilkesM. S.DayF. C.BellD. S.HigaJ. K. (2006). The roulette wheel: An aid to informed decision making. PLoS Medicine, 3, 743–748.
12.
HoffmanR. R.DetweilerM.ConwayJ. A.LiptonK. (1993). Some considerations in using color in meteorological displays. Weather and Forecasting, 8, 505–518.
13.
Johnson-LairdP. N. (2005). Mental models and thought. In HolyoakJ.MorrisonR. G. (Eds.), The Cambridge handbook of thinking and reasoning (pp. 185–208). New York: Cambridge University Press.
14.
JoslynS.JonesD.Strategies in naturalistic decision-making: A cognitive task analysis of naval weather forecasting. In SchraagenJ. M. (Ed.), Naturalistic decision making and macrocognition. Burlington, VT: Ashgate Publishing. Available at http://www.stat.washington.edu/MURI/.
15.
JoslynS.PakK.JonesD.PylesJ.HuntE. (2007). The effect of probabilistic information on threshold forecasts. Weather and Forecasting, 22, 804–812.
16.
KeithR. (2003). Optimization of value of aerodrome forecasts. Weather and Forecasting, 18, 808–824.
17.
KirschenbaumS. S.ArrudaJ. E. (1994). Effects of graphic and verbal probability information on command decision making. Human Factors, 36, 406–418.
18.
KleinG. (1998). Sources of power: How people make decisions. Cambridge, MA: MIT Press.
19.
KoehlerD. J.BrennerL.GriffinD. (2002). The calibration of expert judgment: Heuristics and biases beyond the laboratory. In GilovichT.GriffinD.KahnemanD. (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 686–715). New York: Cambridge University Press.
20.
KoriatA.LichtensteinS.FischhoffB. (1980). Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory, 6(2), 107–118.
21.
LevinI. P.SchneiderS. L.GaethG. J. (1998). All frames are not created equal: A typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76(2), 149–188.
22.
LichtensteinS.FischhoffB.PhillipsL. D. (1982). Calibration of probabilities: The state of the art to 1980. In KahnemanD.SlovicP.TverskyA. (Eds.), Judgment under uncertainty: Heuristics and biases (pp. 306–334). Cambridge, UK: Cambridge University Press.
23.
MacEachrenA. M. (1992). Visualizing uncertain information. Cartographic Perspectives, 13, 10–19.
24.
MurphyA. H.WinklerR. L. (1974a). Credible interval temperature forecasting: Some experimental results. Monthly Weather Review, 162, 784–794.
25.
MurphyA. H.WinklerR. L. (1974b). Subjective probability forecasting experiments in meteorology: Some preliminary results. Bulletin of the American Meteorological Society, 55, 1206–1216.
26.
MurphyA. H.WinklerR. L. (1977). Can weather forecasters formulate reliable forecasts of precipitation and temperature? National Weather Digest, 2(2), 2–9.
27.
National Research Council (NRC). (2006). Completing the forecast: Characterizing and communicating uncertainty for better decisions using weather and climate forecasts. Washington, DC: National Academies Press.
28.
NichollsN. (1999). Cognitive illusions, heuristics, and climate prediction. Bulletin of the American Meteorological Society, 80, 1385–1397.
29.
NormanD. A. (1986). Cognitive engineering. In NormanD. A.DraperS. W. (Eds.), User centered system design: New perspectives on human-computer interaction (pp. 31–61). Mahwah, NJ: Erlbaum.
30.
OlstonC.MackinlayJ. D. (2002). Visualizing data with bounded uncertainty. Proceedings of the IEEE Symposium on Information Visualization (pp. 37–40). Los Alamitos, CA: IEEE Press.
31.
PielkeR. A.Jr. (1999). Nine fallacies of floods. Climatic Change, 42, 413–438.
32.
PliskeR. M.KlingerD.HuttonR.CrandallB.KnightB.KleinG. (1997). Understanding skilled weather forecasting: Implications for training and the design of forecasting tools (Tech. Report No. AL/HR-CR-1997–0003 for the Air Force Material Command, Armstrong Laboratory, Human Resources Directorate, Brooks Air Force Base, TX). Fairborn, OH: Klein Associates.
33.
QuayleJ. D.BallL. J. (2000). Working memory, metacognitive uncertainty and belief bias in syllogistic reasoning. Quarterly Journal of Experimental Psychology, 53A, 1202–1223.
34.
RafteryA. E.GneitingT.BalabdaouiF.PolakowskiM. (2005). Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review, 133, 1155–1174.
35.
RoulstonM. S.BoltonG. E.KleitA. N.Sears-CollinsA. L. (2006). A laboratory study of the benefits of including uncertainty information in weather forecasts. Weather and Forecasting, 21, 116–123.
SimonH. A. (1979). Models of thought. New Haven, CT: Yale University Press.
38.
SlovicP. (1987). Perception of risk. Science, 236, 280–285.
39.
TverskyA.KahnemanD. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.
40.
TverskyA.KahnemanD. (1992). Advances in prospect theory: Cumulative representations of uncertainty. Journal of Risk and Uncertainty, 5, 297–323.
41.
VesseyI. (1991). Cognitive fit: A theory-based analysis of the graphs versus tables literature. Decision Sciences, 22, 219–240.
42.
WilsonT. D.HoustonC. E.EtlingK. M.BrekkeN. (1996). A new look at anchoring effects: Basic anchoring and its antecedents. Journal of Experimental Psychology: General, 125, 387–402.
