In this article, we are reporting the findings of a usability study of opinion mining systems’ visualizations. The objectives of this study are to first to rank the visualizations of the opinion mining systems and second to identify important visualization metrics. A questionnaire survey was designed to ask users their level of agreement or disagreement about the 11 selected visualizations against a set of information visualization metrics on a Likert scale. The data were collected by conducting seminars and using a web-based online questionnaire (N = 146). The collected data were analyzed using descriptive statistics and an independent sample t-test to rank the visualizations and investigate differences between perceptions of the two groups of respondents (the participants of the seminars and the online questionnaire), respectively. The results revealed that simple, eye pleasing, easy to understand, user-friendly visualizations with less pre-knowledge required rated higher than others. It is concluded that the participants of the online questionnaire mostly required more pre-knowledge to comprehend the visualizations as compared to the participants of the seminars. The important information visualization metrics are eye pleasing, easy to understand, user-friendly, informative design, usefulness, and representation style. The results of this study could aid in the design and development of visualizations for opinion mining systems.
DaveKWayILawrenceS. Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In: Proceedings of the 12th international conference on World Wide Web (WWW ’03), 2003, pp. 519–528. New York: ACM Press, http://doi.acm.org/10.1145/775152.775226
2.
WuYWeiFLiuS. OpinionSeer: interactive visualization of hotel customer feedback. IEEE T Vis Comput Gr2010; 16(6): 1109–1118.
3.
TurneyPDLittmanML. Measuring praise and criticism: inference of semantic orientation from association. ACM Trans Inform Syst2003; 21(4): 315–346.
4.
PangBLeeL. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of 42nd meeting of the Association for Computational Linguistics (ACL ‘04), Barcelona, 21–26 July 2004, pp. 271–278. Association for Computational Linguistics.
5.
GregoryMLChinchorNWhitneyP. User-directed sentiment analysis: visualizing the affective content of documents. In: Proceedings of the workshop on sentiment and subjectivity in text, Sydney, NSW, Australia, July 2006, pp. 23–30. Stroudsburg, PA: ACL.
6.
WannerFRohrdantzCMansmannF. Visual sentiment analysis of RSS news feeds featuring the US presidential election in 2008. In: Workshop on visual interfaces to the social and the semantic web (VISSW 2009), Sanibel Island, FL, 8 February 2009.
7.
ChenCIbekwe-sanjuanFSanjuanE. Visual analysis of conflicting opinions. In: Proceedings of the IEEE symposium on visual analytics and technology, Baltimore, MD, 31 October–2 November 2006, pp. 59–66. IEEE.
8.
GamonMBasuSBelenkoD. BLEWS: using blogs to provide context for news articles. In: Proceedings of the international conference on weblogs and social media, Seattle, WA, 30 March–2 April 2008.
9.
KhanMKhanSS. Data and information visualization methods, and interactive mechanisms: a survey. Int J Comput Appl2011; 34(1): 1–14.
10.
OsimoDMuredduF. Research challenge on opinion mining and sentiment analysis. In: Using open data: policy modeling, citizen empowerment, data journalism, 2012, http://www.w3.org/2012/06/pmod/opinionmining.pdf
11.
MorinagaSYamanishiKTateishiK. Mining product reputations on the Web. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining, Edmonton, AB, Canada, 23–25 July 2002, pp. 341–349. New York: ACM Press.
12.
MiaoQLiQDaiR. AMAZING: a sentiment mining and retrieval system. Expert Syst Appl2009; 36(3): 7192–7198.
13.
XuKLiaoSSLiJ. Mining comparative opinions from customer reviews for competitive intelligence. Decis Support Syst2011; 50(4): 743–754.
14.
GamonMAueACorston-OliverS. Pulse: mining customer opinions from free text. In: International symposium on intelligent data analysis, 2005, pp. 121–132, http://www.springerlink.com/index/94q1nrhfc8a4e8tn.pdf (accessed 12 December 2011).
15.
OelkeDHaoMRohrdantzC. Visual opinion analysis of customer feedback data. In: Proceedings of the IEEE symposium on visual analytics science and technology, Atlantic City, NJ, 11–16 October 2009, pp. 187–194. IEEE.
16.
LiuBHuMChengJ. Opinion observer: analyzing and comparing opinions on the Web. In: Proceedings of the 14th international conference on World Wide Web (WWW ’05), 2005, pp. 342–351. New York: ACM Press, http://doi.acm.org/10.1145/1060745.1060797
17.
DraperGMLivnatYRiesenfeldRF. A survey of radial methods for information visualization. IEEE T Vis Comput Gr2009; 15(5): 759–776.
18.
NightingaleF. Notes on matters affecting the health, efficiency, and hospital administration of the British army: founded chiefly on the experience of the late war. London: Harrison and Sons, 1858.
19.
FreitasCMDSLuzzardiPRGCavaRA. On evaluating information visualization techniques. In: Proceedings of advanced visual interfaces, Trento, 2002, pp. 2–3.
20.
ISO 9241-10:1996. Ergonomic requirements for office work with visual display terminals (VDTs). Part 10: dialogue principles.
21.
ISO 9241-11:1994. Ergonomic requirements for office work with visual display terminals (VDTs). Part 11: guidance on usability. Draft international standard.
22.
MayhewDJ. Principles and guidelines in software user interface. Upper Saddle River, NJ: Prentice Hall, 1992.
23.
SmithSLMosierJN. Design guidelines for user-system interface software, MA: USAF Electronic Systems Division, Hanscom Air Force Base, Massachusetts, USA, 1984.
24.
OulanovAPajarilloEJY. CUNY+ web: usability study of the web-based GUI version of the bibliographic database of the City University of New York (CUNY). Electron Libr2002; 20(6): 481–487.
25.
BrinckTGergleDWoodSD. Usability for the web: designing web sites that work. San Francisco, CA: Morgan Kaufmann, 2002.
26.
MIT Information Services and Technology Department. Usability guidelines, 2December2004.
27.
LeeKP. A study on the improvement plan by analyzing user interaction pattern with the RISS. Technical report KR2004-17, 2004. Seoul, South Korea: KERIS.
BoothP. Introduction into human-computer interaction. London: Lawrence Erlbaum Associates, 1989.
30.
ZviranMGlezerCAvniI. User satisfaction from commercial web sites: the effect of design and use. Inf Manag2006; 43(2): 157–178.
31.
ScholtzJ. Beyond usability: evaluation aspects of visual analytic environments. In: Proceedings of the IEEE symposium on visual analytics science and technology, Baltimore, MD, 31 October–2 November 2006, pp. 145–150. New York: IEEE.
32.
JeongWHanHH. Usability study on newspaper mobile websites. OCLC Syst Serv2012; 28(4): 180–198.
33.
YushianaMRaniWA. Heuristic evaluation of interface usability for a web-based OPAC. Libr Hi Tech2007; 25(4): 538–549.
34.
KimHHKimYH. Usability study of digital institutional repositories. Electron Libr2008; 26(6): 863–881.
35.
CasalóLVFlaviánCGuinalíuM. The role of satisfaction and website usability in developing customer loyalty and positive word-of-mouth in the e-banking services. Int J Bank Market2008; 26(6): 399–417.
36.
RavendranRMacCollIDochertyM. Usability evaluation of a tag-based interface. J Usability Stud2012; 7(4): 143–160.
37.
FeiziAWongCY. Usability of user interface styles for learning a graphical software application. In: 2012 International conference on computer & information sciences, Kuala Lumpur, Malaysia, 12–14 June 2012, pp. 1089–1094. New York: IEEE.
38.
Van RielACRLiljanderVJurriënsP. Exploring consumer evaluations of e-services: a portal site. Int J Serv Ind Manag2012; 12(4): 359–377.
39.
DudekDMastoraALandoniM. Is Google the answer? A study into usability of search engines. Libr Rev2007; 56(3): 224–233.
40.
FreitasCMDSLuzzardiPRGCavaRA. Evaluating usability of information visualization techniques. In: CHI 2002—5th workshop on human factors in computer systems, 2002.
41.
ZukTSchlesierLNeumannP. Heuristics for information visualization evaluation. In: Proceedings of the 2006 AVI workshop on beyond time and errors: novel evaluation methods for information visualization, Venice, 23 May 2006.
42.
ZukTCarpendaleS. Theoretical analysis of uncertainty visualizations. In: Proceedings of the SPIE-IS&T electronic imaging (ed ErbacherRFRobertsJCGröhnMT.), San Jose, CA, 15 January 2006, vol. 6060. Bellingham, WA: SPIE.
43.
AmarRStaskoJ. A knowledge task-based framework for design and evaluation of information visualizations. In: Proceedings of the IEEE symposium on information visualization (INFOVIS ‘04), Austin, TX, 10–12 October 2004, pp. 143–150. New York: IEEE.
44.
ShneidermanB. The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the IEEE symposium on visual languages, Boulder, CO, 3–6 September 1996, pp. 336–343. New York: IEEE.
45.
WehrendSLewisC. A problem-oriented classification of visualization techniques. In: Proceedings of the first IEEE conference on visualization, San Francisco, CA, 23–26 October 1990, pp. 139–143. New York: IEEE.
46.
ScholtzJ. Developing qualitative metrics for visual analytic environments. In: Proceedings of the 3rd BELIV ’10 workshop: beyond time and errors: novel evaluation methods for information visualization, Atlanta, GA, 10–15 April 2010, pp. 1–7. New York: ACM Press.
47.
ForsellCJohanssonJ. An heuristic set for evaluation in information visualization. In: Proceedings of the international conference on advanced visual interfaces (AVI ’10), Rome, 25–29 May 2010, pp. 199–206. New York: ACM Press.
48.
BrathR. Metrics for effective information visualization. In: Proceedings of the IEEE symposium on information visualization, Phoenix, AZ, 18–25 October 1997, pp. 108–111. New York: IEEE Computer Society.
49.
ConnellTAOChoongY. Metrics for measuring human interaction with interactive visualizations for information analysis. In: Proceedings of ACM CHI 2008 conference on human factors in computing systems, Florence, 5–10 April 2008, pp. 1493–1496. New York: ACM Press.
50.
PlaisantCFeketeJ-DGrinsteinG. Promoting insight-based evaluation of visualizations: from contest to benchmark repository. IEEE T Vis Comput Gr2007; 14(1): 120–134.
51.
BaiXWhiteDSundaramD. Purposeful visualization. In: Proceedings of the 44th Hawaii international conference on system sciences, Kauai, HI, 4–7 January 2011, pp. 1–10. New York: IEEE Computer Society.
52.
TufteER. The visual display of quantitative information, 2001. DOI: 10.1016/S0140-6736(05)70412-8.
53.
BertinJ. Semiology of graphics. Madison, WI: The University of Wisconsin Press, 1983.
54.
WareC. Information visualization: perception for design. 2nd ed.San Francisco, CA: Morgan Kaufmann Publishers, 2004.
55.
WissUCarrDJonssonH. Evaluating three-dimensional information visualization designs: a case study of three designs. In: Proceedings of the IEEE conference on information visualization, London, 29–31 July 1998, pp. 137–144. New York: IEEE Computer Society.
LamHBertiniEIsenbergP. Empirical studies in information visualization: seven scenarios. IEEE Trans Vis Comput Graph2012; 18: 1520–1536.
58.
ScholtzJPlaisantCWhitingM. Evaluation of visual analytics environments: the road to the visual analytics science and technology challenge evaluation methodology. Inf Vis2014; 13(4): 326–335.
59.
BrathR. 3D Interactive information visualization: guidelines from experience and analysis of applications. In: Proceedings of HCI international ‘97, San Francisco, CA, 24–29 August 1997, pp. 1–5. ELSEVIER.
60.
PillatRMValiatiERAFreitasCMDS. Experimental study on evaluation of multidimensional information visualization techniques. In: Proceedings of the 2005 Latin American conference on human-computer interaction (CLIHC ’05), Cuernavaca, Mexico, 23–26 October 2005, pp. 20–30. New York: ACM Press.
61.
PretoriusMCCalitzAPGreunenVD. The added value of eye tracking in the usability evaluation of a network management tool. In: Proceedings of the 2005 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries (SAICSIT ’05), White River, South Africa, 2005, pp. 1–10. New York: ACM Press.
62.
KangY-AGörgCStaskoJ. How can visual analytics assist investigative analysis? Design implications from an evaluation. IEEE T Vis Comput Gr2010; 17(5): 570–583.
63.
KeimDAHaoMCDayalU. Hierarchical pixel bar charts. IEEE T Vis Comput Gr2002; 8(3): 255–269.
64.
VliegenRvan WijkJJven der LindenE-J. Visualization business data with generalized treemaps. IEEE T Vis Comput Gr2006; 12(5): 789–796.
65.
KeimDAHaoMCDayalU. Value-cell bar charts for visualizing large transaction data sets. IEEE T Vis Comput Gr2007; 13(4): 822–833.
66.
KrattJStrobeltHDeussenO. Improving stability and compactness in street layout visualizations. In: Proceedings of the vision, modeling, and visualization workshop 2011, Berlin, 4–6 October 2011, pp. 285–292. Mitti, Germany: Eurographics Association.
67.
StaskoJZhangE. Focus+context display and navigation techniques for enhancing radial, space-filling hierarchy visualizations. In: IEEE symposium on information visualization, Salt Lake City, UT, 9–10 October 2000, pp. 57–65. New York: IEEE.
68.
ElmqvistNDragicevicPFeketeJ-D. Rolling the dice: multidimensional visual exploration using scatterplot matrix navigation. IEEE T Vis Comput Gr2008; 14(6): 1141–1148.
69.
HaoMCDayalUSharmaRK. Visual analytics of large multi-dimensional data using variable binned scatter plots. In: Proceedings of visualization and data analysis (ed ParkJHaoMCWongPC.), San Jose, CA, 17 January 2010, vol. 7530. Bellingham, WA: SPIE.
