The VAST Challenge: history,scope,and outcomes: An introduction to the Special Issue

Abstract

The annual Visual Analytics Science and Technology (VAST) challenge provides Visual Analytics researchers, developers, and designers an opportunity to apply their best tools and techniques against invented problems that include a realistic scenario, data, tasks, and questions to be answered. Submissions are processed much like conference papers, contestants are provided reviewer feedback, and excellence is recognized with awards. A day-long VAST Challenge workshop takes place each year at the IEEE VAST conference to share results and recognize outstanding submissions. Short papers are published each year in the annual VAST proceedings. Over the history of the challenge, participants have investigated a wide variety of scenarios, such as bioterrorism, epidemics, arms smuggling, social unrest, and computer network attacks, among many others. Contestants have been provided with large numbers of realistic but synthetic Coast Guard interdiction records, intelligence reports, hospitalization records, microblog records, personal RFID tag locations, huge amounts of cyber security log data, and several hours of video. This paper describes the process for developing the synthetic VAST Challenge datasets and conducting the annual challenges. This paper also provides an introduction to this special issue of Information Visualization, focusing on the impacts of the VAST Challenge.

Keywords

Visual analytics evaluation contest sensemaking critical thinking

Get full access to this article

View all access options for this article.

References

Thomas

Cook

(eds). Illuminating the path: the research and development agenda for visual analytics. IEEE Press, 2005.

University of Maryland. Visual analytics benchmark repository, http://hcil.cs.umd.edu/localphp/hcil/vast/archive/provenance.php (accessed 20 January 2013).

Institute for Intelligence Studies at Mercyhurst. http://www.iismu.org/ (accessed 24 May 2013).

Pacific Northwest National Laboratory. IN-SPIRE™ visual document analysis, http://in-spire.pnnl.gov/ (accessed 31 October 2012).

Berthold

Cebron

Dill

. KNIME: the Konstanz Information Miner. In: Preisach

Burkhardt

Schmidt-Thieme

(eds) Data analysis, machine learning and applications. Berlin: Springer, 2008, pp. 319–326.

Whiting

Haack

Varley

. Creating realistic, scenario-based synthetic data for test and evaluation of information analytics software. In: Proceedings of the 2008 Workshop on BEyond time and errors: novel evaluation methods for Information Visualization April 5, 2008, a publication of conference CHI’08 CHI Conference on Human Factors in Computing Systems, Florence, Italy, 5–10 April 2008 New York, NY: ACM.

Roberts

. You can write a mystery (You can write it). Cincinnati, OH: Writer’s Digest Books, 1999.

Bell

Whaley

. Cheating and deception. New Brunswick, NJ: Transaction Publishers, 1991.

Masud-Piloto

. From welcomed exiles to illegal immigrants: Cuban migration to the US, 1959–1995. New York: Rowman & Littlefield Publishers, Inc., 1996.

10.

Zimmerman

. The black market for wildlife: combating transnational organized crime in the illegal wildlife trade. Vanderbilt J Transnatl Law 2003; 36: 1657–1689.

11.

Amico

Kocka

. Information assurance visualizations for specific stages of situational awareness and intended uses: lessons learned. In: Visualization for computer security, Minneapolis, 2005, pp. 107–112. IEEE.

12.

Meitzler

Ouderkirk

Hughes

. Security assessment simulation toolkit. Report, Pacific Northwest National Laboratory, PNNL-19864, November 2009. Pacific Northwest National Laboratory (PNNL).

13.

Brigantic

Ebert

Corley

. Development of a quick look pandemic influenza modeling and visualization tool. In: ISCRAM, Seattle, WA, 2–5 May 2010.

14.

Tisue

Wilensky

. NetLogo: a simple environment for modeling complexity. In: International conference on complex systems, Boston, MA, 16–21 May 2004

15.

Plaisant

Fekete

Grinstein

Promoting insight based evaluation of visualizations: from contest to benchmark repository. IEEE T Vis Comput Gr 2008; 14: 120–134.

16.

Zheng

Dai

KDD CUP-2005 report: facing a great challenge. ACM SIGKDD Explor 2005; 7: 91–99.

17.

Grinstein

O’Connell

Laskowski

. VAST 2006 contest: a tale of alderwood. In: Visual analytics science and technology, Baltimore, MD, 29 October-3 November 2006, pp. 215–216. Piscataway, NJ: IEEE Computer Society Press.

18.

Grinstein

Plaisant

Laskowski

. VAST 2007 contest: Blue Iguanodon. In: Visual analytics science and technology, Sacramento, CA, 28 October-1 November 2007, pp. 231–232. Piscataway, NJ: IEEE Computer Society Press.

19.

Grinstein

Plaisant

Laskowski

. VAST 2008 Challenge: introducing the mini-challenges. In: Visual analytics science and technology, Columbus, Ohio, 19–24 October 2008, pp. 195–196. Piscataway, NJ: IEEE Computer Society Press.

20.

Grinstein

Scholtz

Whiting

. VAST 2009 Challenge: an insider threat. In: Visual analytics science and technology, Atlantic City, NJ, 11–16 October 2009, pp. 243–244. Piscataway, NJ: IEEE Computer Society Press.

21.

Grinstein

Konecni

Scholtz

. VAST 2010 Challenge: arms dealing and pandemics. In: Visual analytics science and technology, Salt Lake City, UT, 24–29 October 2010, pp. 263–264. Piscataway, NJ: IEEE Computer Society Press.

22.

Grinstein

Cook

Havig

. VAST 2011 Challenge: cyber security and epidemic. In: Visual analytics science and technology, Providence, RI, 23–28 October 2011. Piscataway, NJ: IEEE Computer Society Press.

23.

Cook

Grinstein

Whiting

. VAST Challenge Seattle, WA, 14–19 October 2012: visual analytics for big data. In: Visual analytics science and technology, 2012, pp. 251–255. Piscataway, NJ: IEEE Computer Society Press.

24.

Visual Analytics Community. VAST Challenge 2012 sponsorship call, http://www.vacommunity.org/VAST+Challenge+2012+Sponsorship+Call (2012, accessed 11 November 2012).

25.

Palantir. VAST 2010 Challenge, http://www.palantir.com/2010/12/vast2010/ (2010, accessed 11 November 2012).

26.

Costello

Grinstein

Plaisant

. Advancing user-centered evaluation of visual analytic environments through contests. Inform Visual 2009; 8: 230–238.

27.

Evfimievski

Grandison

. Privacy-preserving data mining. In: Ferraggine

Doorn

Rivero

(eds) Handbook of Research on Innovations in Database Technologies and Applications: Current and Future Trends. Hershey, PA: Information Science Reference, 2009, pp. 527–536.

28.

Dasgupta

Kosara

. Adaptive privacy-preserving visualization using parallel coordinates. IEEE T Vis Comput Gr 2011; 17: 2241–2248.