A Simulation Study of Computer-Supported Inferential Analysis Under Data Overload

Abstract

A simulation study of inferential analysis under data overload was conducted with professional intelligence analysts. Using a process tracing methodology, patterns in information sampling and sources of inaccurate statements were identified when analysts were asked to analyze something outside their base of expertise, were tasked with a tight deadline, and had a large data set. The main contribution from this study is a better understanding of potential vulnerabilities in inferential analysis in challenging situations. These vulnerabilities are informative because they point to a set of design criteria that human-centered solutions to data overload should meet in order to be useful. These evaluation criteria are interesting, in part, because they are so difficult to address. They are not amenable to simple, straightforward adjustments or feature additions to current tools. Meeting these design criteria will require innovative design concepts.

Get full access to this article

View all access options for this article.

References

Josephson

(1994). Abductive Inference. Cambridge University Press, New York, NY.

Schum

D.A.

(1994). The Evidential Foundations of Probabilistic Reasoning. New York: John Wiley and Sons.

Woods

D. D.

(1993). Process tracing methods for the study of cognition outside of the experimental psychology laboratory. In Klein

Orasanu

Calderwood

(Eds.), Decision Making in Action: Models and Methods. Ablex Publishing Corporation, Norwood, NJ.

Woods

D. D.

Watts

J. C.

(1997). How not to have to navigate through too many displays. In Helander

(Ed.), Handbook of Human-Computer Interaction, 2nd edition. Elsevier Science Publishers, B.V., North Holland.

Woods

D.D.

Patterson

E.S.

Roth

E.M.

(1998). Aiding the Intelligence Analyst in Situations of Data Overload: A Diagnosis of Data Overload. Institute for Ergonomics/Cognitive Systems Engineering Laboratory Report, ERGO-CSEL 98-TR-03, The Ohio State University, Columbus OH.

Woods

D.D.

Patterson

E.S.

Roth

E.M.

Christoffersen

(1999). Can we ever escape from data overload? A cognitive systems diagnosis. Proceedings of the Human Factors and Ergonomics Society 43^rd annual meeting. Houston, TX.