Sage Journals: Discover world-class research

Abstract

Language Model is used to describe and calculate the probability of a reasonable sentence occurrence in natural language. In practical applications, language model as the core of natural language processing is often used in machine translation, information indexing, voice recognition, context processing such as sentiment recognition and other tasks. We will discuss advantages and weaknesses of traditional statistical language models and neural Network Language Models such as CBOW and Skip-gram. Keeping in view the traditional statistical language model and neural network model, we will try to put forward the word vector model based on part of speech and sentiment information (PSWV-model) in order to use more natural language information such as word order features, part of speech features, and sentiment polarity information under the framework of Mikolov’s model. And finally we will present our deliberations on some advantages of PSWV model and other models including CBOW and Skip-Gram, CDNV in the NLP tasks including named entities recognition and sentiment polarity analysis.

Keywords

Deep learning word vector sentiment analysis named entity recognition

Get full access to this article

View all access options for this article.

References

, Lin

, Meng

, et al., Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks, Algorithms 9(2) (2016), 41–49.

Tang

, Wei

, Qin

, Yang

, Liu

and Zhou

, Sentiment embeddings with applications to sentiment analysis, IEEE Transactions on Knowledge and Data Engineering 28(2) (2016), 496–509.

Liang

, Liu

, Tan

and Bai

, Sentiment classification based on as-lda model, Procedia Computer Science 31 (2014), 511–516.

Hinton

G.E.

and Salakhutdinov

R.R.

, Reducing the dimensionality of data with neural networks, Science 313(5786) (2006), 504–507.

Bengio

, Ducharme

, Vincent

, et al., A neural probabilistic language, Journal of Machine Learning Research 3(2) (2003), 1137–1155.

Sainath

T.N.

, Kingsbury

and Saon

, Deep Convolutional Neural Networks for Large-scale Speech Tasks, Neural Networks 64 (2014), 39–48.

Henry

, Cuffy

and Mcinnes

B.T.

, Vector representations of multi-word terms for semantic relatedness, Journal of Biomedical Informatics 77 (2018), 111.

Baotian

, Buzhou

and etc, A novel word embedding learning model using the dissociation between nouns and verbs, Neurocomputing 2016(171) (2016), 1108–1117.

Maas

A.L.

, Daly

R.E.

, Pham

P.T.

, et al., Learning word vectors for sentiment analysis, Proceedings ComTonal Linguistics of the 49th Annual Meeting of the Association for Human Language Technologies-Volume 1, Association for Computational Linguistics 2011 (2011), 142–150.

10.

XiangJie

, Zhiliang

and Wansen

, Emotional Modeling in an E-learning System Based on OCC Theory, Computer Science 37(5) (2010), 214–218.

11.

, Chen

, Xia

, Lu

, Liu

and Wang

, Word embedding composition for data imbalances in sentiment and emotion classification, Cognitive Computation 7(2) (2015), 226–240.

12.

Collobert

, Weston

, Bottou

, et al., Natural Language Processing (almost) from Scratch, Journal of Machine Learning Research 12 (2011), 2493–2537.

13.

Wang

and Manning

C.D.

, Learning a product of experts with elitist lasso, Newdesign.aclweb.org 34(1) (2013), 58–65.

14.

Graves

and Schmidhuber

, Frame wise phoneme classification with bidirectional LSTM and other neural network architectures, Neural Networks 18(5) (2005), 602–610.

The deep learning word vector model using part of speech and sentiment information

Abstract

Keywords

Get full access to this article

References