Complex data-driven decisions often need complex data. Existing advanced visualizations, such as parallel coordinates and treemap, can help present this information but they are generally considered overly complex to read for everyday decision makers. Prior research on training techniques that reported training time included video or interactive tools that may not be fast and easy enough for decision makers to learn at the time of use. Therefore, inspired by learning theories in psychology and preferences of decision makers, we designed a set of on-the-fly training techniques based on static views to help them quickly learn and leverage these visualizations in the decision-making process. To evaluate the effectiveness of these training techniques, we conducted an empirical study. We found some evidence that three of our training techniques could help reduce the perception of difficulty in reading at least one of the complex visualizations. However, for each visualization, these training techniques were only effective in improving accuracy or response time in some of the tasks. Finally, we identified readability challenges in these two complex visualizations that could inform the design of future training techniques.
SvenssonRBTaghavianfarM.Toward becoming a data-driven organization: Challenges and benefits. In: Lecture notes in business information processing, 2020; 385 LNBIP: 3–19. DOI: 10.1007/978-3-030-50316-1_1.
JohnsonBShneidermanB. Tree-maps: a space-filling approach to the visualization of hierarchical information structures. In: 1991 Proceeding visualization, 1991, pp. 284–291. IEEE Computer Society. DOI: 10.1109/VISUAL.1991.175815
DimaraEBezerianosADragicevicP.Conceptual and methodological issues in evaluating multidimensional visualizations for decision support. IEEE Trans Vis Comput Graph2018; 24(1): 749–759.
8.
DyBIbrahimNPoorthuisA, et al. Improving visualization design for effective multi-objective decision making. IEEE Trans Vis Comput Graph2022; 28(10): 3405–3416.
9.
PerkhoferLMHoferPWalchshoferC, et al. Interactive visualization of big data in the field of accounting: a survey of current practice and potential barriers for adoption. J Appl Account Res2019; 20: 497–525.
KwonBCLeeB. A comparative evaluation on online learning approaches using parallel coordinate visualization. In: Proceedings of the 2016 CHI conference on human factors in computing systems, CHI’16, 2016, pp. 993–997. Association for Computing Machinery. DOI: 10.1145/2858036.2858101
12.
RuchikachornPMuellerK.Learning visualizations by analogy: Promoting visual literacy through visualization morphing. IEEE Trans Vis Comput Graph2015; 21(9): 1028–1044.
13.
KalinaCPowellKC.Cognitive and social constructivism: developing tools for an effective classroom. Education2009; 130: 241–250.
14.
LewisD.Objectivism vs. constructivism: The origins of this debate and the implications for instructional designers. Development of Technology-Based Instruction, 2001.
15.
OlsonMH.Introduction to theories of learning. Routledge, 2015.
16.
DimaraEZhangHToryM, et al. The unmet data visualization needs of decision makers within organizations. IEEE Trans Vis Comput Graph2022; 28(12): 4101–4112.
17.
AndrienkoGAndrienkoN. Constructing parallel coordinates plot for problem solving. In: 1st International symposium on smart graphics, 2001, pp. 9–14.
18.
SiirtolaHRäihäKJ.Interacting with parallel coordinates. Interact Comput2006; 18: 1278–1309.
19.
AsahiTTuroDShneidermanB.Visual decision-making: using treemaps for the analytic hierarchy process. In Conference companion on human factors in computing systems, CHI ’95, 1995, pp. 405–406. Association for Computing Machinery. DOI: 10.1145/223355.223747
20.
FiratEEJoshiALarameeRS.Interactive visualization literacy: the state-of-the-art. Inf Vis2022; 21(3): 285–310.
21.
EdgeDLarsonJMobiusM, et al. Trimming the hairball: edge cutting strategies for making dense graphs usable. In: 2018 IEEE international conference on Big Data (Big Data), 2018, pp. 3951–3958. IEEE. DOI: 10.1109/BigData.2018.8622521
22.
VelezMSilverDTremaineM.Understanding visualization through spatial ability differences. In: VIS 05, IEEE Visualization, 2005, pp. 511–518. IEEE. DOI: 10.1109/VISUAL.2005.1532836
23.
WojtonMAHaydeDHeimlichJE, et al. Begin at the beginning: a constructionist model for interpreting data visualizations. Curator2018; 61(4): 559–574.
24.
LeeSKimSHKwonBC.Vlat: Development of a visualization literacy assessment test. IEEE Trans Vis Comput Graph2017; 23(1): 551–560.
25.
AlperBRicheNHChevalierF, et al. Visualization literacy at elementary school. In: Proceedings of the 2017 CHI conference on human factors in computing systems, 2017, pp. 5485–5497.
BennettCRyallJSpalteholzL, et al. The aesthetics of graph visualization. In: CunninghamDWMeyerGNeumannL (eds) Computational aesthetics in graphics, visualization, and imaging. The Eurographics Association, 2007. https://diglib.eg.org/items/8f9c6297-fddf-4ed0-84e3-df3ef6858de5
28.
PurchaseHC.Metrics for graph drawing aesthetics. J Vis Lang Comput2002; 13: 501–516.
29.
SugiyamaK.Graph drawing and applications for software and knowledge engineers. World Scientific, 2002.
30.
DunneCShneidermanB. Motif simplification: Improving network visualization readability with fan, connector, and clique glyphs. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’13, 2013, pp. 3247–3256. Association for Computing Machinery. DOI: 10.1145/2470654.2466444
31.
MonroeMLanRLeeH, et al. Temporal event sequence simplification. IEEE Trans Vis Comput Graph2013; 19: 2227–2236.
32.
FiratEEDenisovaAWilsonML, et al. P-lite: A study of parallel coordinate plot literacy. Vis Inform2022; 6(3): 81–99.
33.
O’HandleyBJWuYDuanH, et al. Treevisual: design and evaluation of a web-based visualization tool for teaching and learning tree visualization. In: Proceedings of American Society for Engineering Education Annual Conference, 2022.
34.
StoiberCWalchshoferCGrassingerF, et al. Design and comparative evaluation of visualization onboarding methods. In: Proceedings of the 14th International symposium on visual information communication and interaction, VINCI ’21, 2021. Association for Computing Machinery. DOI: 10.1145/3481549.3481558
35.
FiratEELangCSrinivasB, et al. A constructivism-based approach to treemap literacy in the classroom. In: MaganaAZaraJ (eds) Eurographics 2023 - education papers. The Eurographics Association, 2023. https://diglib.eg.org/items/c652afb4-3361-49a0-b8ec-32b7079aaa87
36.
GonzálezVMMarkG. “Constant, constant, multi-tasking craziness”: managing multiple working spheres. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’04, 2004, pp. 113–120. Association for Computing Machinery. DOI: 10.1145/985692.985707
37.
MarkGVoidaSCardelloA. “A pace not dictated by electrons”: an empirical study of work without email. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’12, 2012, pp. 555–564. Association for Computing Machinery. DOI: 10.1145/2207676.2207754.
38.
VrasidasC.Constructivism versus objectivism: Implications for interaction, course design, and evaluation in distance education. Int J Educ Telecommun2000; 6(4): 339–362.
39.
NunesMBMcPhersonM.Constructivismvs.Objectivism: where is difference for designers of e-learning environments? In: Proceedings 3rd IEEE international conference on advanced technologies, 2003, pp. 496–500. IEEE.
40.
RasemanWJJacobsonJKasprzykJR.Parasol: an open source, interactive parallel coordinates library for multi-objective decision making. Environ Model Softw2019; 116: 153–163.
41.
CibulskiLMitterhoferHMayT, et al. Paved: Pareto front visualization for engineering design. Comput Graph Forum2020; 39(3): 405–416.
42.
FiedlerCScheibelWLimbergerD, et al. Survey on user studies on the effectiveness of treemaps. In: Proceedings of the 13th international symposium on visual information communication and interaction, 2020, pp. 1–10. DOI: 10.1145/3430036.3430054.
43.
SaatyTL.The analytic hierarchy process: planning, priority setting, resource allocation. McGraw-Hill International Book Company, 1980.
44.
GuoPJKimJRubinR. How video production affects student engagement: an empirical study of mooc videos. In: Proceedings of the first ACM conference on Learning@ scale conference, 2014, pp. 41–50.
45.
EliasMAufaureMABezerianosA. Storytelling in visual analytics tools for business intelligence. In: Human-computer interaction–INTERACT 2013: 14th IFIP TC 13 international conference, Cape Town, South Africa, 2–6 September 2013, pp. 280–297, Proceedings, Part III 14. Springer.
46.
FraserR.Reorganizing the organization chart. Public Adm Rev1978; 38(3): 280–282.
47.
AguinisHVillamorIRamaniRS.Mturk research: review and recommendations. J Manag2021; 47(4): 823–837.
48.
LuLNealeNLineND, et al. Improving data quality using amazon mechanical turk through platform setup. Cornell Hosp Q2022; 63(2): 231–246.
49.
HeerJBostockM. Crowdsourcing graphical perception: Using mechanical turk to assess visualization design. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’10, 2010, pp. 203–212. Association for Computing Machinery. DOI: 10.1145/1753326.1753357.
50.
BorgoRMicallefLBachB, et al. Information visualization evaluation using crowdsourcing. Comput Graph Forum2018; 37: 573–595.
51.
MauriMElliTCavigliaGet al. Rawgraphs: A visualisation platform to create open outputs. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter. CHItaly’ 17, New York, NY: Association for Computing Machinery. DOI:10.1145/3125571.3125585.
52.
AmarREaganJStaskoJ. Low-level components of analytic activity in information visualization. In: IEEE Symposium on Information Visualization, INFOVIS 2005, 2005, pp. 111–117. DOI: 10.1109/INFVIS.2005.1532136
53.
DimaraEStaskoJ.A critical reflection on visualization research: Where do decision making tasks hide?IEEE Trans Vis Comput Graph2022; 28(1): 1128–1138.
54.
DragicevicP. Fair statistical communication in HCI. Springer International Publishing, Cham, 2016, pp. 291–330. DOI:10.1007/978-3-319-26633-6 13.
55.
HonickeTBroadbentJFuller-TyszkiewiczM.The self-efficacy and academic performance reciprocal relationship: the influence of task difficulty and baseline achievement on learner trajectory. High Educ Res Dev2023; 42(8): 1936–1953.
56.
JacksonJW.Enhancing self-efficacy and learning performance. J Exp Educ2002; 70(3): 243–254.
57.
RaoofiSTanBHChanSH.Self-efficacy in second/foreign language learning contexts. Engl Lang Teach2012; 5(11): 60–73.
58.
EllenPSBeardenWOSharmaS.Resistance to technological innovations: an examination of the role of self-efficacy and performance satisfaction. J Acad Mark Sci1991; 19: 297–307.
59.
KimHWKankanhalliA.Investigating user resistance to information systems implementation: a status quo bias perspective. MIS Q2009; 33(3): 567–582. http://www.jstor.org/stable/20650309
60.
KitturAChiEHSuhB.Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI conference on human factors in computing systems, 2008, pp. 453–456.
61.
BuchananEMScofieldJE.Methods to detect low quality data and its implication for psychological research. Behav Res Methods2018; 50: 2586–2596.
62.
ZhaoJLiuZDontchevaM, et al. Matrixwave: visual comparison of event sequence data. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, 2015, pp. 259–268.
63.
WiseJAThomasJJPennockK, et al. Visualizing the non-visual: spatial analysis and interaction with information from text documents. In: Proceedings of visualization 1995 Conference, 1995, pp. 51–58. IEEE.
64.
HayesDKovalerchukB. Parallel coordinates for discovery of interpretable machine learning models. In: KovalerchukBNazemiKAndonieR, et al. (eds) Artificial intelligence and visualization: advancing visual knowledge discovery. Springer Nature, 2024, pp. 125–158.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
