Critical analyses provide information visualization practitioners with insight into the range and suitability of different techniques for visualization. Theory provides the necessary models and vocabulary to deconstruct, explain and classify visualizations, allowing the analysis and comparison of alternate designs, and evaluation of their success. While the critical vocabulary for information visualization in general is well developed, the same cannot be said for ‘hybrid’ information visualizations which combine abstract representation of data with figurative elements such as illustrations. Figurative elements are widely used in information visualization in practice and are increasingly recognized as beneficial for memorability. However, the information encoded by a figurative image and how that information contributes to the overall content of the visualization lacks robust definition within visualization theory. To support critical analysis of hybrid visualization, we provide a model of the information content of a figurative image, which we call the figurative frame model. We use the model to classify hybrid visualizations along two dimensions: information density in the images (defined as the number of features and preserved measurements) and integration of figurative and abstract forms of representation. The new vocabulary for analysing hybrid visualizations reveals how the figurative images expand the expressiveness of information visualization by integrating descriptive and abstract information and allows the formulation of new measures of visualization quality which can be applied to hybrid visualizations.
CanhamMHegartyM. Effects of knowledge and display design on comprehension of complex graphics. Learn Instr2010; 20(2): 155–166.
2.
TufteER. The visual display of quantitative information. 2nd ed.Cheshire, CT: Graphics Press, 2001.
3.
KosslynSM. Understanding charts and graphs. Appl Cognitive Psych1989; 3(3): 185–225.
4.
WongsuphasawatKMoritzDAnandAet al. Voyager: exploratory analysis via faceted browsing of visualization recommendations. IEEE T Vis Comput Gr2016; 22(1): 649–658.
5.
BatemanSMandrykRLGutwinCet al. Useful junk?: the effects of visual embellishment on comprehension and memorability of charts. In: Proceedings of the SIGCHI conference on human factors in computing systems, Atlanta, GA, 10–15 April 2010. New York: ACM.
6.
LiHMoacdiehN. Is ‘chart junk’ useful? An extended examination of visual embellishment. Proc Hum Fact Erg Soc Annu Meet2014; 58: 1516–1520.
7.
BorgoRAbdul-RahmanAMohamedFet al. An empirical study on using visual embellishments in visualization. IEEE T Vis Comput Gr2012; 18(12): 2759–2768.
8.
BorkinMABylinskiiZKimNWet al. Beyond memorability: visualization recognition and recall. IEEE T Vis Comput Gr2016; 22(1): 519–528.
9.
BorkinMVoABylinskiiZet al. What makes a visualization memorable?IEEE T Vis Comput Gr2013; 19(12): 2306–2315.
SpritzerASBoyJDragicevicPet al. (eds). Towards a smooth design process for static communicative node-link diagrams. Comput Graph Forum2015; 34: 461–470.
12.
LeeBKaziRHSmithG. SketchStory: telling more engaging stories with data through freeform sketching. IEEE T Vis Comput Gr2013; 19(12): 2416–2425.
13.
BachBKerracherNHallKWet al. Telling stories about dynamic networks with graph comics. In: Proceedings of the conference on human factors in information systems (CHI), Santa Clara, CA, 7–12 May 2016. New York: ACM.
14.
ByrneLAngusDWilesJ. Acquired codes of meaning in data visualization and infographics: beyond perceptual primitives. IEEE T Vis Comput Gr2016; 22(1): 509–518.
15.
HullmanJDiakopoulosN. Visualization rhetoric: framing effects in narrative visualization. IEEE T Vis Comput Gr2011; 17(12): 2231–2240.
16.
SegelEHeerJ. Narrative visualization: telling stories with data. IEEE T Vis Comput Gr2010; 16: 1139–1148.
17.
HullmanJDruckerSRicheNHet al. A deeper understanding of sequence in narrative visualization. IEEE T Vis Comput Gr2013; 19(12): 2406–2415.
18.
HullmanJAdarEShahP. Benefitting InfoVis with visual difficulties. IEEE T Vis Comput Gr2011; 17(12): 2213–2222.
19.
KosslynSM. Understanding charts and graphs. Appl Cogn Psychol1989; 3(3): 185–225.
20.
SkauDHarrisonLKosaraR. An evaluation of the impact of visual embellishments in bar charts. Comput Graph Forum2015; 34: 221–230.
21.
KimNWSchweickartELiuZet al. Data-driven guides: supporting expressive design for information graphics. IEEE T Vis Comput Gr2017; 1: 491–500.
22.
MarkDM. Human spatial cognition. In: Medyckyj-ScottD (ed.) Human factors in geographical information systems. London: Belhaven Press, 1993, pp. 51–60.
23.
NeurathO. International picture language; the first rules of isotype: with isotype pictures. London: K. Paul, Trench, Trubner & Company, 1936.
24.
ChevalierFVuillemotRGaliG. Using concrete scales: a practical framework for effective visual depiction of complex measures. IEEE T Vis Comput Gr2013; 19(12): 2426–2435.
25.
JangAMacLeanDLHeerJ. BodyDiagrams: improving communication of pain symptoms through drawing. In: Proceedings of the SIGCHI conference on human factors in computing systems, Toronto, ON, Canada, 26 April–1 May 2014. New York: ACM.
26.
BarnardM. Graphic design as communication. New York: Routledge, 2013.
27.
ArnheimR. Visual thinking. Berkelay, CA: University of California Press, 1969.
28.
TsafrirJOhryA. Medical illustration: from caves to cyberspace. Health Info Libr J2001; 18(2): 99–109.
29.
InnisRE. Semiotics: an introductory anthology. Bloomington, IN: Indiana University Press, 1985.
30.
FreedmanEGShahP. Toward a model of knowledge-based graph comprehension. In International conference on theory and application of diagrams, 18 April 2002, pp. 18–30. Berlin, Heidelberg: Springer.
31.
HegartyM. The cognitive science of visual-spatial displays: implications for design. Top Cogn Sci2011; 3(3): 446–474.
32.
HarozSKosaraRFranconeriSL. Isotype visualization: working memory, performance, and engagement with pictographs. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, Seoul, Korea, 18–23 April 2015. New York: ACM.
ZiemkiewiczCKosaraR. Embedding information visualization within visual representation. In: RasZRibarskyW (eds) Advances in information and intelligent systems (Studies in Computational Intelligence, 251). Berlin: Springer, 2009, pp. 307–326.
WilkinsonL. The grammar of graphics. New York: Springer, 2006.
37.
CardSMackinlayJShneidermanB. Readings in information visualization: using vision to think. San Francisco, CA: Morgan Kaufmann Publishers, 1999.
38.
KosaraR. Visualization criticism – the missing link between information visualization and art. In: 11th international conference on information visualization, 2007 IV ’07, Zurich, 4–6 July 2007.
39.
BartlettFC. Remembering: a study in experimental and social psychology. Cambridge: Cambridge University Press, 1932.
40.
SchoonmakerD. Sunshine on a cloudy day. Am Sci2006; 94(3): 217.
ClevelandWSMcGillR. An experiment in graphical perception. Int J Man Mach Stud1986; 25(5): 491–500.
43.
KehrerJHauserH. Visualization and visual analysis of multifaceted scientific data: a survey. IEEE T Vis Comput Gr2013; 19(3): 495–513.
44.
SchulzHJ. Treevis.net: a tree visualization reference. IEEE Comput Graph2011; 31(6): 11–15.
45.
AlsallakhBMicallefLAignerWet al. Visualizing sets and set-typed data: state-of-the-art and future challenges. In: BorgoRMMaciejewskiRViolaI (eds) Eurographics conference on visualization (EuroVis), Swansea, 9–13 June 2014.
46.
BachBDragicevicPArchambaultDet al. A review of temporal data visualizations based on space-time cube operations. In: BorgoRMaciejewskiRViolaI (eds) Eurographics conference on visualization (EuroVis), Swansea, 9–13 June 2014.
47.
AignerWMikschSSchumannHet al. Survey of visualization techniques. In: AignerWMikschSSchumannHet al. (eds) Visualization of time-oriented data (Human-Computer Interaction Series). London: Springer, 2011, pp. 147–254.
48.
LeeSKimS-HHungY-Het al. How do people make sense of unfamiliar visualizations?: a grounded model of Novice’s information visualization sensemaking. IEEE T Vis Comput Gr2016; 22(1): 499–508.
49.
BörnerKMalteseABallietRNet al. Investigating aspects of data visualization literacy using 20 information visualizations and 273 science museum visitors. Inform Visual2016; 15: 198–213.
50.
LeeMDReillyREButaviciusME. An empirical evaluation of Chernoff faces, star glyphs, and spatial visualizations for binary data. In: Proceedings of the Asia-Pacific symposium on information visualisation, vol. 24, Adelaide, SA, Australia, 2003. Darlinghurst, NSW, Australia: Australian Computer Society, Inc.
51.
ChernoffH. The use of faces to represent points in k-dimensional space graphically. J Am Stat Assoc1973; 68(342): 361–368.
52.
WoodM. Visualization in historical context. In: MacEachrenAMTaylorDRF (eds) Visualization in modern cartography. Amsterdam: Elsevier, 2013, pp. 13–26.
53.
LinSFortunaJKulkarniCet al. Selecting semantically-resonant colors for data visualization. Comput Graph Forum2013; 32: 401–410.
54.
DorlingDFairbairnD. Mapping: ways of representing the world. London: Routledge, 2013.
55.
MackanessWA. Integration and evaluation of map generalization. In: ButtenfieldBPMcMasterRB (eds) Map generalization: making rules for knowledge representation. London: Longman Group, 1991, pp. 217–226.
56.
BrehmerMMunznerT. A multi-level typology of abstract visualization tasks. IEEE T Vis Comput Gr2013; 19(12): 2376–2385.
57.
ArtimoK. The bridge between cartographic and geographic information systems. In: MacEachrenAMRuxtonDTaylorF (eds) Visualisation in modern cartography, vol. 2. Amsterdam: Elsevier, 1994, pp. 45–61.
58.
FennaD. Cartographic science: a compendium of map projections, with derivations. Boca Raton, FL: CRC Press, 2006.
59.
SteersJA. An introduction to the study of map projections. London: University of London Press, 1959.
WardJL. The perception of pictorial space in perspective pictures. Leonardo1976; 279–288.
62.
BowerGHKarlinMBDueckA. Comprehension and memory for pictures. Memory & Cognition1975; 3(2): 216–220.
63.
BoukhelifaNBezerianosAIsenbergTet al. Evaluating sketchiness as a visual variable for the depiction of qualitative uncertainty. IEEE T Vis Comput Gr2012; 18(12): 2769–2778.
64.
KellerJ. Why sketch. In: CanfieldMR (ed.) Field notes on science & nature. Cambridge, MA: Harvard University Press, 2011, pp. 161–185.
65.
EnglehardtY. The language of graphics. Amsterdam: University of Amsterdam (UvA), 2002.
66.
WareC. Information visualization: perception for design. San Francisco, CA: Morgan Kaufmann Publishers, 2000.
67.
DayNSimpsonK. Field guide to the birds of Australia. 7th ed.Camberwell, VIC, Australia: Viking, 2004.
68.
EtemadKCarpendaleSSamavatiF. Node-ring graph visualization clears edge congestion. In: Proceedings of the IEEE VIS Arts Program (VISAP), Paris, 9–14 November 2014, pp. 67–74.
ZukTSchlesierLNeumannPet al. Heuristics for information visualization evaluation. In: Proceedings of the 2006 AVI workshop on beyond time and errors: novel evaluation methods for information visualization, Venice, 23 May 2006. New York: ACM.
72.
MackinlayJ. Automating the design of graphical presentations of relational information. ACM T Graphic1986; 5(2): 110–141.
