In the last decade, there has been an exponential growth of asynchronous online conversations (e.g. blogs), thanks to the rise of social media. Analyzing and gaining insights from such discussions can be quite challenging for a user, especially when the user deals with hundreds of comments that are scattered around multiple different conversations. A promising solution to this problem is to automatically mine the major topics from conversations and organize them into a hierarchical structure. However, the resultant topic hierarchy can be noisy and/or it may not match the user’s current information needs. To address this problem, we introduce a novel human-in-the-loop approach that allows the user to revise the topic hierarchy based on her feedback. We incorporate this approach within a visual text analytics system that helps users in analyzing and getting insights from conversations by exploring and revising the topic hierarchy. We evaluated the resulting system with real users in a lab-based study. The results from the user study, when compared to its counterpart that does not support interactive revisions of a hierarchical topic model, provide empirical evidence of the potential utility of our system in terms of both performance and subjective measures. Finally, we summarize generalizable lessons for introducing human-in-the-loop computation within a visual text analytics system.
CareniniGMurrayGNgR. Methods for mining and summarizing text conversations. San Rafael, CA: Morgan & Claypool, 2011.
2.
IndratmoVassileva JGutwinC. Exploring blog archives with interactive visualization. In: Proceedings of the working conference on advanced visual interfaces (AVI), Napoli, 28–30 May 2008, pp. 39–46. New York: ACM.
3.
WeiFLiuSSongY, et al. TIARA: a visual exploratory text analytic system. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining, Washington, DC, 25–28 July 2010, pp. 153–162. New York: ACM.
4.
ChooJLeeCReddyCK, et al. UTOPIAN: user-driven topic modeling based on interactive nonnegative matrix factorization. IEEE T Vis Comput Gr2013; 19(12): 1992–2001.
5.
HuYBoyd-GraberJSatinoffB, et al. Interactive topic modeling. Mach Learn2014; 95(3): 423–469.
6.
HoqueECareniniG. ConVis: a visual text analytic system for exploring blog conversations. Comput Graph Forum2014; 33(3): 221–230.
7.
HoqueECareniniG. MultiConVis: a visual text analytics system for exploring a collection of online conversations. In: Proceedings of the 21st international conference on intelligent user interfaces (IUI), Sonoma, CA, 7–10 March 2016, pp. 96–107. New York: ACM.
8.
DouWYuLWangX, et al. HierarchicalTopics: visually exploring large text collections using topic hierarchies. IEEE T Vis Comput Gr2013; 19(12): 2002–2011.
9.
CuiWLiuSWuZ, et al. How hierarchical topics evolve in large text corpora. IEEE T Vis Comput Gr2014; 20(12): 2281–2290.
10.
LiuSWangXChenJ, et al. TopicPanorama: a full picture of relevant topics. In: Proceedings of the IEEE conference on visual analytics science and technology (VAST), Paris, 25–31 October 2014, pp. 183–192. New York: IEEE.
11.
LeeTYSmithASeppiK, et al. The human touch: how non-expert users perceive, interpret, and fix topic models. Int J Hum: Comput St2017; 105: 28–42.
12.
HoqueECareniniG. ConVisIT: interactive topic modeling for exploring asynchronous online conversations. In: Proceedings of the 20th international conference on intelligent user interfaces (IUI), Atlanta, GA, 29 March–1 April 2015, pp. 169–180. New York: ACM.
13.
MunznerT. A nested model for visualization design and validation. IEEE T Vis Comput Gr2009; 15(6): 921–928.
14.
SedlmairMMeyerMMunznerT. Design study methodology: reflections from the trenches and the stacks. IEEE T Vis Comput Gr2012; 18(12): 2431–2440.
15.
DörkMCarpendaleSCollinsC, et al. VisGets: coordinated visualizations for web-based information exploration and discovery. IEEE T Vis Comput Gr2008; 14(6): 1205–1212.
16.
ViégasFBGolderSDonathJ. Visualizing email content: portraying relationships from conversational histories. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI), Montréal, QC, Canada, 22–27 April 2006, pp. 979–988. New York: ACM.
17.
DörkMGruenDWilliamsonC, et al. A visual backchannel for large-scale events. IEEE T Vis Comput Gr2010; 16(6): 1129–1138.
18.
HavreSHetzlerEWhitneyP, et al. ThemeRiver: visualizing thematic changes in large document collections. IEEE T Vis Comput Gr2002; 8(1): 9–20.
19.
LiuSWangXSongY, et al. Evolutionary Bayesian rose trees. IEEE T Knowl Data En2015; 27(6): 1533–1546.
20.
LiuSYinJWangX, et al. Online visual analytics of text streams. IEEE T Vis Comput Gr2016; 22(11): 2451–2466.
21.
AndrzejewskiDZhuXCravenM. Incorporating domain knowledge into topic modeling via Dirichlet Forest priors. In: Proceedings of the 26th annual international conference on machine learning, Montreal, QC, Canada, 14–18 June 2009, pp. 25–32. New York: ACM.
22.
RamageDHallDNallapatiR, et al. Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of the 2009 conference on empirical methods on natural language processing (EMNLP), Singapore, 6–7 August 2009, pp. 248–256. New York: ACM.
23.
YangYPanSSongY, et al. User-directed non-disruptive topic model update for effective exploration of dynamic content. In: Proceedings of the 20th international conference on intelligent user interfaces (IUI), Atlanta, GA, 29 March–1 April 2015, pp. 158–168. New York: ACM.
24.
JotySCareniniGNgRT. Topic segmentation and labeling in asynchronous conversations. J Artif Intell Res2013; 47: 521–573.
25.
ChuangJHuYJinA, et al. Document exploration with topic modeling: designing interactive visualizations to support effective analysis workflows. In: Proceedings of the NIPS workshop on topic models: computation, application and evaluation, Lake Tahoe, Nevada, 10 December 2013, pp. 1–4.
26.
LeeHKihmJChooJ, et al. iVisClustering: an interactive visual document clustering via topic modeling. Comput Graph Forum2012; 31: 1155–1164.
27.
PirolliPSchankPHearstM, et al. Scatter/gather browsing communicates the topic structure of a very large text collection. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI), Vancouver, BC, Canada, 13–18 April 1996, pp. 213–220. New York: ACM.
28.
ChuangJManningCDHeerJ. Termite: visualization techniques for assessing textual topic models. In: Proceedings of the international working conference on advanced visual interfaces (AVI), Capri Island, 21–25 May 2012, pp. 74–77. New York: ACM.
29.
PaateroPTapperU. Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics1994; 5(2): 111–126.
30.
AlberdiESleemanDH. ReTAX: a step in the automation of taxonomic revision. Artif Intell1997; 91(2): 257–279.
31.
NikitinaNRudolphSGlimmB. Interactive ontology revision. Web Semant Sci Serv Agents World Wide Web2012; 12–13: 118–130.
32.
VikramSDasguptaS. Interactive Bayesian hierarchical clustering. In: Proceedings of the 33rd international conference on machine learning, New York, 19–24 June 2016, pp. 2081–2090. New York: ACM.
33.
GalleyMMcKeownKFosler-LussierE, et al. Discourse segmentation of multi-party conversation. In: Proceedings of the 41st annual meeting on association for computational linguistics (ACL), Sapporo, Japan, 7–12 July 2003, pp. 562–569. New York: ACM.
34.
ShiJMalikJ. Normalized cuts and image segmentation. IEEE T Pattern Anal2000; 22(8): 888–905.
35.
NewmanMEGirvanM. Finding and evaluating community structure in networks. Phys Rev E2004; 69(2): 026113.
36.
ZhouDOrshanskiySAZhaH, et al. Co-ranking authors and documents in a heterogeneous network. In: Proceedings of the 7th IEEE international conference on data mining, Omaha, NE, 28–31 October 2007, pp. 739–744. New York: IEEE.
37.
KirosRZhuYSalakhutdinovRR, et al. Skip-thought vectors. In: Proceedings of the advances in neural information processing systems, Montréal, QC, Canada, 7–12 December 2015, pp. 3294–3302. New York: Curran Associates, Inc.
38.
GoffinPWillettWFeketeJD, et al. Exploring the placement and design of word-scale visualizations. IEEE T Vis Comput Gr2014; 20(12): 2291–2300.
39.
WillettWHeerJAgrawalaM. Scented widgets: improving navigation cues with embedded visualizations. IEEE T Vis Comput Gr2007; 13(6): 1129–1136.
40.
SteinbergerMWaldnerMStreitM, et al. Context-preserving visual links. IEEE T Vis Comput Gr2011; 17(12): 2249–2258.
41.
HeerJRobertsonGG. Animated transitions in statistical data graphics. IEEE T Vis Comput Gr2007; 13(6): 1240–1247.
42.
FleissJLLevinBPaikMC. Statistical methods for rates and proportions. Hoboken, NJ: John Wiley & Sons, 2013.
43.
McKennaSMazurDAgutterJ, et al. Design activity framework for visualization design. IEEE T Vis Comput Gr2014; 20(12): 2191–2200.
44.
HasanKSNgV. Automatic keyphrase extraction: a survey of the state of the art. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (ACL), Baltimore, MD, 23–25 June 2014, pp. 1262–1273. Canada: ACL.
45.
ChuangJRamageDManningC, et al. Interpretation and trust: designing model-driven visualizations for text analysis. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI), Austin, TX, 5–10 May 2012, pp. 443–452. New York: ACM.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
