Accuracy of Intelligence Forecasts From the Intelligence Consumer’s Perspective

Abstract

Accurate forecasting is a vital part of intelligence assessment. Only recently has intelligence forecast accuracy been quantitatively tracked. Mandel and Barnes reported on a long-term study of intelligence forecasts that examined accuracy from the analysts’ perspective using numerical probabilities that were not reported to intelligence consumers. The present research reassessed the accuracy of those forecasts from an intelligence consumer’s perspective using findings from an experiment that elicited from subjects’ numerical probability equivalents for the linguistic probabilities that consumers would have read in intelligence reports. Forecast accuracy was undiminished when assessed from the consumers’ perspective (inferred from subjects’ median numerical equivalents) because the intended meaning of the probability terms used by the intelligence unit corresponded well to the average meaning assigned by subjects. The findings also showed that interpretations of linguistic probabilities are context-dependent. Linguistic probabilities were discriminated better when applied to outcomes that represented successes rather than failures.