The beneficial role of the VAST Challenges in the evolution of GeoTime and nSpace2

Abstract

Oculus participated in the first VAST Challenge in 2006 and in every subsequent year except one. Although we have won 5 awards in the last 7 years, they are not the main reason we spend much time and effort preparing our submission each year. The VAST Challenge has proven to be a valuable exercise in evaluating and refining our tools and our thinking. No organization that has the desire to create the best possible analytic tools should pass by the unique opportunity to work on a realistic set of tasks, against nontoy datasets containing ground truth, and to be evaluated by both senior analysts and experts within the visual analytics community. Oculus staff and products have benefited from the experience as have the whole visual analytics community researchers and practitioners.

Keywords

Visual analytics human information interaction sense making geospatial information systems evaluation contest

Get full access to this article

View all access options for this article.

References

Kapler

Wright

. GeoTime information visualization. In: IEEE symposium on information visualization (InfoVis 2004), Austin, TX, 10–12 October 2004, pp. 25–32.

Wright

Schroh

Proulx

. the sandbox for analysis—concepts and methods. In: SIGCHI conference on human factors in computing systems, Montreal, ON, Canada, 24–27 April 2006, pp. 801–810. New York: ACM.

Redish

. Expanding usability testing to evaluate complex systems. J Usabil Stud 2007; 2(3): 102–111.

Costello

Grinstein

Plaisant

. Advancing user-centered evaluation of visual analytic environments through contests. Inform Visual 2009; 8(3): 230–238.

Kapler

Eccles

Harper

. Configurable spaces: temporal analysis in diagrammatic contexts. In: IEEE symposium on visual analytics science and technology (VAST ’08), Columbus, OH, 19–24 October 2008, pp. 43–50.

Ware

. Information visualization: perception for design. 3rd ed. San Francisco, CA: Morgan Kaufmann Publishers Inc., 2000, p. 346.

Jonker

Wright

Proulx

. Information triage with TRIST. In: Intelligence analysis conference, Washington, DC, 2–4 May 2005.

Johnson

. Designing with the mind in mind: simple guide to understanding user interface design rules. Burlington, MA: Morgan Kaufmann Publishers, 2010, p. 161.

SEMVAST Project. Visual analytics benchmark repository, http://hcil.cs.umd.edu/localphp/hcil/vast/archive/viewbm.php

10.

Plaisant

Fekete

Grinstein

. Promoting insight based evaluation of visualizations: from contest to benchmark repository. IEEE T Vis Comput Gr 2008; 14(1): 120–134.