Hierarchical text classification of a Web taxonomy is challenging because it is a very large-scale problem with hundreds of thousands of categories and associated documents. Furthermore, the conceptual levels and training data availabilities of categories vary widely. The narrow-down approach is the state of the art; it utilizes a search engine for generating candidates from the taxonomy and builds a classifier for the final category selection. In this paper, we take the same approach but address the issue of using global information in a language modelling framework to improve effectiveness. We propose three methods of using non-local information for the task: a passive way of utilizing global information for smoothing; an aggressive way where a top-level classifier is built and integrated with a local model; and a method of using label terms associated with the path from a category to the root, which is based on our systematic observation that they are underrepresented in the documents. For evaluation, we constructed a document collection from Web pages in the Open Directory Project. A series of experiments and their results show the superiority of our methods and reveal the role of global information in hierarchical text classification.
BroderAFontouraMJosifovskiVRiedelL. A semantic approach to contextual advertising. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2007, pp. 559–566.
2.
BroderACiccoloPGabrilovichEJosifovskiVMetzlerDRiedelL. Online expansion of rare queries for sponsored search. In: Proceedings of the 18th international conference on World Wide Web. New York: ACM, 2009, pp. 511–520.
3.
ZhangBLiHLiuYJiLXiWFanW. Improving web search results using affinity graph. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. New York: ACM, 2005, pp. 504–511.
4.
CaiLZhouGLiuKZhaoJ. Large-scale question classification in cQA by leveraging Wikipedia semantic knowledge. In: Proceedings of the 20th ACM international conference on information and knowledge management. New York: ACM, 2011, pp. 1321–1330.
5.
ChenYXueG-RYuY. Advertising keyword suggestion based on concept hierarchy. In: Proceedings of the 2008 international conference on web search and data mining. New York: ACM, 2008, pp. 251–260.
6.
BroderAZFontouraMGabrilovichEJoshiAJosifovskiVZhangT. Robust classification of rare queries using web knowledge. In: Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2007, pp. 231–238.
7.
McCallumARosenfeldRMitchellTMNgAY. Improving text classification by shrinkage in a hierarchy of classes. In: Proceedings of the fifteenth international conference on machine learning. San Francisco, CA: Morgan Kaufmann, 1998, pp. 359–367.
8.
CaiLHofmannT. Hierarchical document categorization with support vector machines. In: Proceedings of the thirteenth ACM conference on information and knowledge management – CIKM ’04, 2004, pp. 78–87.
9.
SebastianiF.Machine learning in automated text categorization. ACM Computing Surveys2002; 34: 1–47.
10.
SunALimE-P. Hierarchical text classification and evaluation. In: Proceedings of the 2001 IEEE international conference on data mining. Washington, DC: IEEE Computer Society, 2001, pp. 521–528.
11.
BennettPNNguyenN. Refined experts: improving classification in large taxonomies. In: Proceedings of the 32nd international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2009, pp. 11–18.
12.
MalikH. Improving hierarchical SVMs by hierarchy flattening and lazy classification. In: Large-scale hierarchical classification workshop (ECIR 2010), 2010.
13.
LiuT-YYangYWanHZengH-JChenZMaW-Y. Support vector machines classification with a very large-scale taxonomy. SIGKDD Explor Newsletter2005; 7: 36–43.
14.
XueG-RXingDYangQYuY. Deep classification in large-scale text hierarchies. In: Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2008, pp. 619–626.
15.
SillaCNJrFreitasAA. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery2011; 22: 31–72.
16.
OhH-SChoiYMyaengS-H. Text classification for a large-scale taxonomy using dynamically mixed local and global models for a node. In: Proceedings of the 33rd European conference on advances in information retrieval, 2011, pp. 7–18.
17.
OhH-SChoiYMyaengS-H. Combining global and local information for enhanced deep classification. In: Proceedings of the 2010 ACM symposium on applied computing, 2010, pp. 1760–1767.
18.
KollerDSahamiM. Hierarchically classifying documents using very few words. In: Proceedings of the 4th international conference on machine learning, 1997, pp. 170–178.
19.
RobertsonSEWalkerS. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In: Proceedings of the 17th annual international ACM SIGIR conference on research and development in information retrieval. New York: Springer, 1994, pp. 232–241.
20.
PonteJMCroftWB. A language modeling approach to information retrieval. In: Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 1998, pp. 275–281.
21.
LaffertyJZhaiC. Document language models, query models, and risk minimization for information retrieval. In: Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2001, pp. 111–119.
22.
KurlandOLeeL. PageRank without hyperlinks: Structural re-ranking using links induced by language models. In: Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2005, pp. 306–313.
23.
ZhaiCLaffertyJ. A study of smoothing methods for language models applied to ad hoc information retrieval. In: Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2001, pp. 334–342.
24.
KurlandOLeeL. Corpus structure, language models, and ad hoc information retrieval. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2004, pp. 194–201.
25.
LiuXCroftWB. Cluster-based retrieval using language models. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. New York: ACM, 2004, pp. 186–193.
26.
TaoTWangXMeiQZhaiC. Language model information retrieval with document expansion. In: Proceedings of the main conference on human language technology conference of the North American Chapter of the Association of Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2006, pp. 407–414.
27.
BaiJSongDBruzaPNieJ-YCaoG. Query expansion using term relationships in language models for information retrieval. In: Proceedings of the 14th ACM international conference on information and knowledge management. New York: ACM, 2005, pp. 688–695.
28.
WeiXCroftWB. Modeling term associations for ad-hoc retrieval performance within language modeling framework. In: Proceedings of the 29th European conference on IR research. Berlin: Springer, 2007, pp. 52–63.
29.
PengFSchuurmansDWangS.Augmenting naive Bayes classifiers with statistical language models. Information Retrieval2004; 7: 317–345.
30.
PengFSchuurmansD.Combining naive Bayes and n-gram language models for text classification. In: Proceedings of the 25th European conference on IR research. Berlin: Springer, 2003, pp. 335–350.
31.
TanSWangYWuG.Adapting centroid classifier for document categorization. Expert Systems with Applications2011; 38: 10264–10273.
32.
BrantsTPopatACXuPOchFJDeanJIncG. Large language models in machine translation. In: Proceedings of the conference on empirical methods in natural language processing, 2007, pp. 858–867.
