The development of intelligent decision support systems requires much research effort to solve decision-making problems’ complexity. In fact, the combination of both intelligent components and visualization aspects in intelligent decision support system required a lot of efforts in order to develop advanced information visualization schemes for decision-making processes. For this, an efficient evaluation of these systems has become a major concern for applications in multiple fields. The reports of the existing usability evaluation studies are helpful to verify the potential and the limitations of these tools. However, it is important to integrate more relevant metrics for visual analytics tasks in dynamic intelligent decision support system. The proposed method consists of a questionnaire that is given to the users and a subsequent analysis of the resulting data using fuzzy logic. The advantage of the fuzzy model is its ability to transform the input survey scores into linguistic variables, as well as linguistic evaluation of the overall intelligent decision support system visualization tool. With this approach, it is possible to model the vagueness in the ordinal judgments obtained from the users’ evaluation about the visualizations of intelligent decision support system and to support uncertainty in such evaluation.
PlaisantC. The challenge of information visualization evaluation. In: AVI ’04 proceedings of the working conference on advanced visual interfaces, Gallipoli, 25–28 May 2004, pp.109–116. New York: ACM.
3.
ShneidermanB. The eyes have it: a task by data type taxonomy for information visualizations. In: VL ‘96 proceedings of the 1996 IEEE symposium on visual languages, Boulder, Colorado, USA, 3–6 September 1996, pp.336–343. Washington, DC: IEEE.
4.
ClevelandWS. The elements of graphing data. Summit, NJ: Hobart Press, 1994.
5.
MazzaRBerreA. Focus group methodology for evaluating information visualization techniques and tools. In: IV ’07 11th international conference on information visualization, Zurich, 4–6 July 2007, pp.74–80. New York: IEEE.
6.
BahloulRBen AyedMAlimiA-M.Vers une method d’évaluation de Système Interactif d’Aide à la Décision basé sur le processus d’ECD. In: Atelier EvalECD’2010, Hammamet, Tunisia, 26–29 January 2010, pp.1–12. Toulouse, France: Cépaduès-éditions.
7.
BrahmiALtifiHBen AyedM. Approach for the evaluation of a KDD based DSS visual representations. In: 2014 middle east conference on biomedical engineering (MECBME), Doha, Qatar, 17–20 February 2014.
WilliamSBuckleyJJ.Fuzzy expert systems and fuzzy reasoning. 1st ed. Hoboken, NJ: Wiley-Interscience, 2005.
10.
PhamBStreitABrownR.Visualisation of information uncertainty: progress and challenges. In: LiereRAdriaansenTZudilova-SeinstraE (eds) Trends in interactive visualization. London: Springer, 2009, pp.19–48.
11.
ChenKC.Decision support system for tourism development: system dynamic approach. J Comput Inform Syst2004; 45(1): 104–112.
12.
NielsenJ.Usability engineering. San Francisco, CA: Morgan Kaufmann, 1993.
13.
KeimDAndrienkoGFeketeJ-Det al. Visual analytics: definition, process, and challenges. In: KerrenAStaskoJ-TFeketeJ-Det al. (eds) Information visualization (Vol. 4950, Lecture Notes in Computer Science). Berlin; Heidelberg: Springer-Verlag, 2008, pp.154–175.
14.
HeerJPererA.A system for modeling, transformation and visualization of multidimensional heterogeneous networks. Infor Vis2014; 13(2): 111–133.
15.
KimHSUnatDBadenSBet al. A new approach to interactive viewpoint selection for volume data sets. Infor Vis2013; 12(3–4): 240–256.
16.
JorgeAPoçasJAzevedoP.A post-processing environment for browsing large sets of association rules. In: Proceedings of workshop on integration of decision support and data mining (IDDM 02), associated with ECML/PKDD 2002 (ed. DMladenicBKavsekNLavracet al.), Helsinki, 19–23 August 2002, pp.53–64. Berlin: Springer-Verlag Press.
17.
AignerWMikschSSchumannHet al. Visualization of time-oriented data. Berlin: Springer, 2011.
18.
KriglsteinSPohlMSmucM. Pep up your time machine: recommendations for the design of information visualizations of time-dependent data. In: HuangW (ed.) Handbook of human centric visualization. Berlin: Springer, 2014, pp.203–225.
19.
MackinlayJDHanrahanPStolteC.Show me: automatic presentation for visual analysis. IEEE T Vis Comput Gr2007; 13(6): 1137–1144.
20.
ShneidermanB.Interactive information visualization to explore and query electronic health records. Foundations Trends Hum: Comput Interact2013; 5(3): 207–298.
21.
FayyadUMDjorgovskiSGWeirN. Automating the analysis and cataloging of sky surveys. In: FayyadUMPiatetsky-ShapiroGSmythPet al. (eds) Advances in knowledge discovery and data mining. Menlo Park, CA: The MIT Press, 1996, pp.471–494.
22.
ShyurH-JJouCChangK.A data mining approach to discovering reliable sequential patterns. J Syst Software2013; 86(8): 2196–2203.
23.
NematiHBarkoCD.Enhancing enterprise decisions through organizational data mining. J Comput Inform Syst2002; 42(4): 21–28.
24.
HeinrichsJHLimJS.Integrating web-based data mining tools with business models for knowledge management. Decis Support Syst2003; 35(1): 103–112.
25.
BoseRSugumaranV.Application of intelligent agent technology for managerial data analysis and mining. Data Base Adv Inf Sy1999; 30(1): 77–94.
26.
PoleseGTroianoMTortoraG.A data mining based system supporting tactical decisions. In: Proceedings of the 14th international conference on software engineering and knowledge engineering, Ischia, 15–19 July 2002, pp.681–684. New York: ACM.
27.
LeeLParkC.Agent and data mining based decision support system and its adaptation to a new customer centric electronic commerce. Expert Syst Appl2003; 25(4): 619–635.
28.
LtifiHKolskiCBen AyedMet al. Human-centered design approach for developing dynamic decision support system based on knowledge discovery in databases. J Decis Syst2013; 22(2): 69–96.
29.
HofmannHSiebesAWilhelmA. Visualizing association rules with interactive mosaic plots. In: KDD ‘00 proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining, Boston, MA, 20–23 August 2000, pp.227–235. New York: ACM.
30.
HanJCerconeN. RuleViz: a model for visualizing knowledge discovery process. In: KDD ‘00 proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining, Boston, MA, 20–23 August 2000, pp.244–253. New York: ACM.
31.
BruzzeseDDavinoC. Visual post-analysis of association rules. In: ECML/PKDD VDM Workshop (LNAI Series), Helsinki, 19–23 August 2002, pp.55–66. Berlin: Springer.
32.
ZhaoKLiuB. Visual analysis of the behavior of discovered rules. In: Proceedings of workshop on visual data mining ACM SIGKDD international conference on knowledge discovery & data mining (KDD 2001), San Francisco, CA, 26–29 August 2001, pp.59–64. New York: ACM.
33.
ZhaoKLiuBTirpakTet al. V-Miner: using enhanced parallel coordinates to mine product design and test data. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining, Seattle, WA, 22–25 August 2004, pp.494–502. New York: ACM.
34.
BlanchardJGuilletFBriandH. Exploratory visualization for association rule rummaging. In: KDD-03 workshop on multimedia data mining, Washington, DC, 24–27 August 2003, pp.107–114. New York: ACM.
35.
OngKHOngKLNgWKet al. CrystalClear: active visualization of association rules. In: ICDM-02 workshop on active mining (AM-02), Maebashi TERRSA, Maebashi City, Japan, 9 December 2002, pp.120–125. New York: IEEE.
36.
MaSHellersteinJ.Ordering categorical data to improve visualization. In: INFOVIS-99, San Francisco airport, Hyatt, San Francisco, CA, 24–29 October 1999, pp.15–18. New York: IEEE.
37.
ZhaoKLiuBTirpakTet al. Opportunity map: a visualization framework for fast identification of actionable knowledge. In: Proceeding of CIKM ‘05 proceedings of the 14th ACM international conference on information and knowledge management, Bremen, 31 October–5 November 2005, pp.60–67. New York: ACM.
38.
LtifiHBen MohamedEBen AyedM.Interactive visual KDD based temporal decision support system. Infor Vis2016; 15(1): 31–50.
39.
AnkerstMAnneKTjoelkerRet al. DataJewel: integrating visualization with temporal data mining. In: SimoffSBöhlenMHMazeikaA (eds) Visual data mining: theory, techniques and tools for visual analytics. Berlin; Heidelberg: Springer, 2008, pp.312–330.
40.
MunagaHIeronuttiLChittaroL. CAST: a novel trajectory clustering and visualization tool for spatio-temporal data. In: Proceedings of the first international conference on intelligent human computer interaction (Organized by the Indian Institute of Information Technology), Allahabad, India, 20–23 January 2009, pp.169–175. Berlin: Springer.
41.
Ramírez-RamírezLLGelYRThompsonMet al. A new surveillance and spatio-temporal visualization tool SIMID: SIMulation of Infectious Diseases using random networks and GIS. Comput Methods Programs Biomed2013; 110(3): 455–470.
42.
MittelstädtSHaoMCDayalUet al. Advanced visual analytics interfaces for adverse drug event detection. In: Proceedings of the 2014 international working conference on advanced visual interfaces (AVI 2014), Como, 27–29 May 2014, pp.237–244. New York: ACM.
43.
FischerFStoffelFMittelstädtSet al. Using visual analytics to support decision making to solve the Kronos incident. In: 2014 IEEE conference on visual analytics science and technology (VAST), Paris, 25–31 October 2014, pp.301–302. New York: IEEE.
44.
MittelstädtSWangXEaglinTet al. An integrated in-situ approach to impacts from natural disasters on critical infrastructures. In: IEEE 48th annual Hawaii international conference on system sciences, HICSS-48, Kauai, HI, 5–8 January 2015, pp.1118–1127. Los Alamitos, CA: IEEE.
45.
WareC.Information visualization: perception for design. San Francisco, CA: Morgan Kaufmann, 2004.
46.
HartsonHHixD.Developing user interfaces: ensuring usability through product and process. Hoboken, NJ: John Wiley & Sons, 1993.
47.
IsenbergTIsenbergPChenJet al. A systematic review on the practice of evaluating visualization. IEEE T Vis Comput Gr2013; 19(12): 2818–2827.
48.
KobsaA. An empirical comparison of three commercial information visualization systems. In: INFOVIS ‘01 proceedings of the IEEE symposium on information visualization, San Diego, CA, 22–23 October 2001, pp.123–130. New York: IEEE.
49.
JuarezO.CAEVA: cognitive architecture to evaluate visualization applications. In: Proceedings seventh international conference on information visualization, 2003, London, 18 July 2003, pp.589–595. New York: IEEE.
50.
GonzalesVKobsaA. A workplace study of the adoption of information visualization systems. In: Proceedings of I-KNOW’03: 3rd international conference on knowledge management, Graz, 2–4 July 2003, pp.92–102. Verlag: J.UCS.
51.
RiemanJ.A field study of exploratory learning strategies. ACM T Comput-Hum Int1996; 3: 189–218.
52.
ShneidermanBPlaisantC. Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies. In: BELIV ‘06 proceedings of the 2006 AVI workshop on beyond time and errors: novel evaluation methods for information visualization, Venice, 23–26 May 2006, pp.81–87. New York: ACM.
53.
SaraiyaPNorthCDucaK.An insight-based methodology for evaluating bioinformatics visualizations. IEEE T Vis Comput Gr2005; 11(4): 443–456.
54.
KosaraR.Visualization criticism—the missing link between information visualization and art. In: Proceedings of the 11th international conference on information visualisation (IV), Zurich, Switzerland, 4–6 July 2007, pp.631–636. Los Alamitos, CA: IEEE.
55.
KurzhalsKBoppCFBässlerJet al. Benchmark data for evaluating visualization and analysis techniques for eye tracking for video stimuli. In: Proceedings of the fifth workshop on beyond time and errors: novel evaluation methods for visualization, Paris, 10 November 2014, pp.54–60. New York: ACM.
56.
RaganEDGoodallJR. Evaluation methodology for comparing memory and communication of analytic processes in visual analytics. In: Proceedings of the fifth workshop on beyond time and errors: novel evaluation methods for visualization, Paris, 10 November 2014, pp.27–34. New York: ACM.
BrookeJ.SUS: a quick and dirty usability scale in usability evaluation in industry. London: Taylor & Francis, 1996, pp.189–194.
59.
LamHBertiniEIsenbergPet al. Empirical studies in information visualization: seven scenarios. IEEE T Vis Comput Gr2012; 18(9): 1520–1536.
60.
ChrimesDKushnirukAKitosNRet al. Usability testing of Avoiding Diabetes Thru Action Plan Targeting (ADAPT) decision support for integrating care-based counseling of pre-diabetes in an electronic health record. Int J Med Inform2014; 83(9): 636–647.
61.
FlaviánCGuinalíuMGurreaR.The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inform Manage2006; 43(1): 1–14.
62.
LewisJRSauroJ. The factor structure of the System Usability Scale. In: KurosuM (ed.) Human centered design. Berlin; Heidelberg: Springer, 2009, pp.94–103.
63.
KeimD.Information visualization and visual data mining. IEEE T Vis Comput Gr2002; 8(1): 1–8.
64.
ChouJ-R.Applying fuzzy linguistic preferences to Kansei evaluation. In: International conference on kansei engineering and emotion research, KEER2014, Linköping, 11–13 June 2014, pp.339–349. Linköping: Linköping University Electronic Press.
65.
MamdaniEHAssilianS.An experiment in linguistic synthesis with a fuzzy logic controller. Int J Man Mach Stud1975; 7: 1–13.
66.
ZadehLA.Outline of a new approach to the analysis of complex systems and decision processes. IEEE T Syst Man Cyb1973; 3: 28–44.
67.
LtifiHTrabelsiGBen AyedMet al. Dynamic decision support system based on Bayesian networks, application to fight against the nosocomial infections. Int J Adv Res Artif Intell2012; 1(1): 22–29.
68.
MackinlayJDRobertsonGGCardSK. The perspective wall: detail and context smoothly integrated. In: Proceedings of the SIGCHI conference on human factors in computing systems, New Orleans, LA, 27 April–2 May 1991, pp.173–176. New York: ACM.
69.
PlaisantCMilashBRoseAet al. LifeLines: visualizing personal histories. In: Proceedings of the SIGCHI conference on human factors in computing systems, Vancouver, BC, Canada, April 1996, pp.221–227. New York: ACM.
70.
ElouniJLtifiHBen AyedM. Knowledge visualization model for intelligent dynamic decision-making. In: The 15th international conference on Hybrid Intelligent Systems (HIS 2015), Seoul, South Korea, 16–18 November 2015, pp.223–235. Berlin: Springer.
71.
Sànchez-MarrèMGibertKCabelloAet al. A methodology for the characterization of intelligent environmental decision support systems. In: International environmental modelling and software society (iEMSs), sixth biennial meeting, Leipzig, 21 September 2012.
72.
AkçayAErtekAEBüyüközkanG.Analyzing the solutions of DEA through information visualization and data mining techniques: smartDEA framework. Expert Syst Appl2011; 39: 7763–7775.
73.
ManeKBizonCSchmittCet al. VisualDecisionLinc: a visual analytics approach for comparative effectiveness-based clinical decision support in psychiatry. J Biomed Inform2012; 45: 101–106.
74.
KuruK.Biomedical visual data analysis to build an intelligent diagnostic decision support system in medical genetics. Artif Intell Med2014; 62(2): 105–118.
75.
FarriOPieckiewiczDSRahmanASet al. A qualitative analysis of EHR clinical document synthesis by clinicians. AMIA Annu Symp Proc2012; 2012: 1211–1220.
76.
KannampallilTGFranklinAMishraRet al. Understanding the nature of information seeking behavior in critical care: implications for the design of health information technology. Artif Intell Med2013; 57(1): 21–29.
