In the fast-moving field of Natural Language Processing (NLP), making lexicons is still a method for many text analysis applications. This process of generating lexicons has traditionally used techniques such as semantic matches, word embeddings, and tools like EMPATH. With the arrival of Large Language Models (LLMs) including GPT-3.5, GPT-4 and Mistral 7b 0.1, we have new ways to create lexicons. This study takes a close look at how these older methods stack up against the newer options brought by LLMs. We carried out a detailed analysis, looking at how well different methods could create lexicons, focusing on their precision, scalability, and concluding on how efficiently they can be used in real-world settings. By using standard NLP tasks like document classification, emotion classification and sentiment analysis, this research prove itself on a variety of datasets to test how well the lexicons worked. This discovery, along with others from our study, aims to help professionals and researchers find the best approaches to lexicon creation today, setting the stage for more research in the NLP field.
MohammadSTurneyP. Emotions evoked by common words and phrases: Using mechanical turk to create an emotion lexicon. In: Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text, 2010, June, pp.26–34.
2.
GiunchigliaFShvaikoPYatskevichM. S-Match: an algorithm and an implementation of semantic matching. In: European semantic web symposium, 2004, May, pp.61–75. Springer, Berlin, Heidelberg.
3.
FastEChenBBernsteinMS. Empath: Understanding topic signals in large-scale text. In: Proceedings of the 2016 CHI conference on human factors in computing systems, 2016, May, pp.4647–4657.
4.
LeQMikolovT. Distributed representations of sentences and documents. In International conference on machine learning, 2014, June, pp.1188–1196. PMLR.
5.
McInnesLHealyJMelvilleJ. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426. 2018.
6.
DevlinJChangMWLeeK,et al.Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
7.
RadfordAWuJChildR, et al.Language models are unsupervised multitask learners. OpenAI Blog2019; 1: 9.
8.
JiangAQSablayrollesAMenschA, et al.Mistral 7B. arXiv preprint arXiv:2310.06825. 2023.
9.
VaswaniAShazeerNParmarN, et al.Attention is all you need. Advances in neural information processing systems2017; 30: 5998–6008.
WangBLiakataMZubiagaA, et al.Smile: Twitter emotion classification using domain adaptation. In: CEUR Workshop proceedings, 2016, Vol. 1619, pp.15–21. Sun SITE Central Europe.
12.
SaraviaELiuHCTHuangYH, et al.CARER: Contextualized affect representations for emotion recognition. In: Proceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp.3687–3697.
13.
EkmanP. Basic emotions. Handbook of Cognition and Emotion1999; 98: 16.
MikolovTChenKCorradoG,et al.Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. 2013.
16.
RumelhartDEHintonGEWilliamsRJ. Learning internal representations by error propagation. 1985.
17.
HochreiterSSchmidhuberJ. Long short-term memory. Neural Computation1997; 9: 1735–1780.
18.
TouvronHMartinLStoneK, et al.Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. 2023.
19.
PenedoGMalarticQHesslowD, et al.The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01116. 2023.
20.
QadirARiloffE. Learning emotion indicators from tweets: Hashtags, hashtag patterns, and phrases. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, October, pp.1203–1209.
21.
MuppidiSGorripatiSKKishoreB. An approach for bibliographic citation sentiment analysis using deep learning’. 1 Jan. 2020: 353–362.
22.
HusseinDMEDM. A survey on sentiment analysis challenges. Journal of King Saud University-Engineering Sciences2018; 30: 330–338.
23.
ZhangHGanWJiangB. Machine learning and lexicon-based methods for sentiment classification: A survey. In: 2014 11th web information system and application conference, 2014, September, pp.262–26. IEEE.
24.
PablosAGCuadrosMRigauG. A Comparison of Domain-based Word Polarity Estimation using different Word Embeddings. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), 2016, pp.54–60, Portorož, Slovenia. European Language Resources Association (ELRA).
25.
MustoCSemeraroGPolignanoM. A Comparison of Lexicon-based Approaches for Sentiment Analysis of Microblog Posts. In: DART@ AI* IA, 2014, December, pp.59–68).
26.
TabakFSEvrimV. Comparison of emotion lexicons. In 2016 HONET-ICT, 2016, October, pp.154–158. IEEE.
27.
GriffithsTLSteyversMTenenbaumJB. Topics in semantic representation. Psychological Review2007; 114: 211.
28.
AngelovD. Top2vec: Distributed representations of topics. arXiv preprint arXiv:2008.09470. 2020.
29.
NaderalvojoudBBozkirASSezerEA. Investigation of term weighting schemes in classification of imbalanced texts. In: Lisbon: Proceedings of European Conference on Data Mining (ECDM), 2014, July, pp.15–7.
