To identify what is not there: A definition of missingness patterns and evaluation of missing value visualization

Abstract

While missing data is a commonly occurring issue in many domains, it is a topic that has been greatly overlooked by visualization scientists. Missing data values reduce the reliability of analysis results. A range of methods exist to replace the missing values with estimated values, but their appropriateness often depend on the patterns of missingness. Increased understanding of the missingness patterns and the distribution of missing values in data may greatly improve reliability, as well as provide valuable insight into potential problems in data gathering and analyses processes, and better understanding of the data as a whole. Visualization methods have a unique possibility to support investigation and understanding of missingness patterns by making the missing values and their relationship to recorded values visible. This article provides an overview of visualization of missing data values and defines a set of three missingness patterns of relevance for understanding missingness in data. It also contributes a usability evaluation which compares visualization methods representing missing values and how well they help users identify missingness patterns. The results indicate differences in performance depending on the visualization method as well as missingness pattern. Recommendations for future design of missing data visualization are provided based on the outcome of the study.

Keywords

Data visualization visual analytics missing data incomplete data usability evaluation

Get full access to this article

View all access options for this article.

References

Kandel

Heer

Plaisant

et al . Research directions in data wrangling: visualizations and transformations for usable and credible data. Inform Visual 2011; 10(4): 271–288.

Wong

BLW

Varga

. Black holes, keyholes and brown worms: challenges in sense making. Hum Fac Erg Soc P 2012; 56: 287–291.

Johansson Fernstad

Glen

. Visual analysis of missing data – to see what isn’t there. In: Proceedings of the 2014 IEEE conference on visual analytics science and technology (VAST), Paris, 25–31 October 2014. New York: IEEE.

Kirk

Visualizing zero: how to show something with nothing, 2014, http://blogs.hbr.org/2014/05/visualizing-zero-how-to-show-something-with-nothing/

Rubin

DB.

Inference and missing data. Biometrika 1976; 63(3): 581–592.

Bache

Lichman

UCI machine learning repository, 2013, http://archive.ics.uci.edu/ml

Fielding

Fayers

Ramsay

CR.

Investigating the missing data mechanism in quality of life outcomes: a comparison of approaches. Health Qual Life Out 2009; 7(1): 57.

Djurcilov

Pang

Visualizing sparse gridded data sets. IEEE Comput Graph 2000; 20(5): 52–57.

Twiddy

Cavallo

Shiri

. Restorer: a visualization technique for handling missing data. In: Proceedings of the conference on visualization’94, Washington, DC, 17–21 October 1994, pp. 212–216. Los Alamitos, CA: IEEE Computer Society Press.

10.

Unwin

Hawkins

Hofmann

et al . Interactive graphics for data sets with missing values: MANET. J Comput Graph Stat 1996; 5(2): 113–122.

11.

Theus

Hofmann

Siegl

et al . MANET extensions to interactive statistical graphics for missing values. In: Eurostat (ed.) New techniques and technologies for statistics II. Amsterdam: IOS Press, 1997, pp. 247–259.

12.

Templ

Alfons

Filzmoser

Exploring incomplete data using visualization techniques. Adv Data Anal Classi 2012; 6(1): 29–47.

13.

Brix

miP: multiple imputation plots, 2011, https://mran.microsoft.com/snapshot/2014-11-03/web/packages/miP/miP.pdf

14.

Schulz

Nocke

Heitzler

et al . A systematic view on data descriptors for the visual analysis of tabular data. Inform Visual 2017; 16(3): 232–256.

15.

Carpenter

Goodrich

YS.

migui: graphical user interface to the ‘mi’ package, 2015, https://cran.r-project.org/web/packages/migui/migui.pdf

16.

Gelman

Hill

et al . Multiple imputation with diagnostics (mi) in r: opening windows into the black box. J Stat Softw 2011; 45(2): 1–31.

17.

Honaker

King

Blackwell

Amelia II: a program for missing data. J Stat Softw 2011; 45(7): 1–47.

18.

Swayne

Buja

Missing data in interactive high-dimensional data visualization. Computation Stat 1998; 13(1): 15–26.

19.

Swayne

Lang

Buja

et al . GGobi: evolving from XGobi into an extensible framework for interactive data visualization. Comput Stat Data An 2003; 43(4): 423–444.

20.

Cheng

Cook

Hofmann

et al . Visually exploring missing values in multivariable data using a graphical user interface. J Stat Softw 2015; 68(6): 1–23.

21.

Cedilnik

Rheingans

. Procedural annotation of uncertain information. In: Proceedings visualization 2000 (VIS 2000), Salt Lake City, UT, 8–13 October 2000, pp. 77–84. New York: IEEE.

22.

Xie

Huang

Ward

et al . Exploratory visualization of multivariate data with variable quality. In: Proceedings of the 2006 IEEE symposium on visual analytics science and technology, Baltimore, MD, 31 October–2 November 2006, pp. 183–190. New York: IEEE.

23.

Arbesser

Spechtenhauser

Mühlbacher

et al . Visplause: visual data quality assessment of many time series using plausibility checks. IEEE T Vis Comput Gr 2017; 23(1): 641–650.

24.

Wang

. Visualization of the critical patterns of missing values in classification data. In: Qiu

Leung

Xue

et al . (eds) Advances in visual information systems. Berlin; Heidelberg: Springer, 2007, pp. 267–274.

25.

Kohonen

The self-organizing map. Neurocomputing 1998; 21(1–3): 1–6.

26.

Eaton

Plaisant

Drizd

Visualizing missing data: graph interpretation user study. In: Costabile

Paternò

(eds) Human-computer interaction – INTERACT 2005. Berlin; Heidelberg: Springer, 2005, pp. 861–872.

27.

Wang

Data mining with incomplete data. In: Wang

(ed.) Encyclopedia of data warehousing and mining. 2nd ed.Hershey, PA: IGI Global, 2009, pp. 526–530.

28.

Templ

Filzmoser

Visualization of missing values using the R-package VIM. Research report CS-2008-1, May2008. Vienna: Department of Statistics and Probability Theory, Vienna University of Technology.

29.

Thompson

Blair

Chen

et al . Video game telemetry as a critical tool in the study of complex skill learning. PLoS ONE 2013; 8(9): e75129.

30.

Graziano

Raulin

ML.

Research methods: a process of inquiry. New York: HarperCollins College Publishers, 1993.

31.

BOS Online Survey tool, 2016, https://www.onlinesurveys.ac.uk/

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.03 MB