Text classification is a fundamental task in Nature Language Processing(NLP). However, with the challenge of complex semantic information, how to extract useful features becomes a critical issue. Different from other traditional methods, we propose a new model based on two parallel RNNs architecture, which captures context information through LSTM and GRU respectively and simultaneously. Motivated by the siamese network, our proposed architecture generates attention matrix through calculating similarity between the parallel captured context information, which ensures the effectiveness of extracted features and further improves classification results. We evaluate our proposed model on six text classification tasks. The result of experiments shows that the ABLGCNN model proposed in this paper has the faster convergence speed and the higher precision than other models.
GravesA., FernándezS. and SchmidhuberJ., Bidirectional LSTM networks for improved phoneme classification and recognition, In ICANN, (2005), pp. 799–804.
2.
RushA.M, ChopraS. and WestonJ., A neural attention model for abstractive sentence summarization, In Proceedings of EMNLP, (2015), pp. 379–389.
3.
de Jesus Cardoso CachopoA.M., Improving methods for single-label text categorization. (2007).
4.
PangB. and LeeL., A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd annual meeting on Association for Computational Linguistics, (2004), pp. 271.
5.
PangB. and LeeL., Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceeding of the 43rd Annual Meeting on Association for Computational Linguistics, (2005), pp. 115–124.
6.
KingmaD. and Ba AdamJ., A method for stochastic optimization, In Proceedings of ICLR, (2015).
7.
BahdanauD., ChoK. and BengioY., Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473, (2014).
8.
BahdanauD., ChoK. and BengioY., Neural machine translation by jointly learning to align and translate, In Proceedings of ICLR, (2015).
9.
HintonG.E., SrivastavaN., KrizhevskyA., SutskeverI. and SalakhutdinovR.R., Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580. (2012).
10.
WiebeJ., WilsonT. and CardieC., Annotating Expressions of Opinions and Emotions in Language,(2-3), Language Resources and Evaluation39 (2005), 165–210.
11.
LiJ., LuongM.-T. and JurafskyD., A hierarchical neural autoencoder for paragraphs and documents, In Proceedings of ACL, (2015), pp. 1106–1115.
12.
TurianJ., RatinovL. and BengioY., Word representations: a simple and general method for semi-supervised learning, In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, (2010), pp. 384–394.
13.
ChoK.H.., MerriënboerB.V., BahdanauD. and BengioY., On the properties of neural machine translation: Encoder-decoder approaches, arXiv preprint arXiv:1409.1259,2014.
14.
HeK., ZhangX., RenS. and SunJ., Deep residual learning for image recognition. In arXiv prepring arXiv:1506.01497.2015.
15.
HuM. and LiuB., Mining and Summarizing Customer Reviews. In Proceedings of ACM SIGKDD, 2004.
16.
LuongM.-T., PhamH. and ManningC.D., Effective approaches to attention-based neural machine translation. In Proceedings of EMNLP, (2015), pp. 1412–1421.
17.
PasunuruR., GuoH. and BansalM., Towards improving abstractive summarization via entailment generation. In Proceeding of the workshop on New Frontiers in Summarization, (2017), pp. 27–32.
18.
ZhouP., QiZ., ZhengS., et al., Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max pooling. arXiv preprint arXiv: 1611:06639, (2016).
19.
PenningtonJ., SocherR. and ManningC., Glove: Global Vectors for Word Representation[C]//Conference on Empirical Methods in Natural Language Processing, (2014), 1532–1543.
20.
CollobertR., WestonJ., BopttouL., KarlenM., KavukcugluK. and KuksaP., Natural Language Processing (Almost) from Scratch, Journal of Machine Learning Research, (2011), pp. 2493–2537.
21.
SocherR., PerelyginA., WuJ.Y., ChuangJ., ManningC.D., NgA.Y. and PottsC., Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP), (2013), 1631 pp. 1642. Citeseer.
22.
HochreiterS., SchmidhuberJ., Long short-term memory,(8), Neural Computation9 (1997), 1735–1780.
23.
IoffeS. and SzegedyC., Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, (2015), pp. 448–456.
24.
LaiS., XuL., LiuK. and ZhaoJ., Recurrent convolutional neural networks for text classification. In AAAI, (2015), pp. 2267–2273.
25.
MikolovT., SutskeverI., ChenK., CorradoG.S. and DeanJ., Distributed representations of word and phrases and their compositionality, In Proceedings of NIPS, (2013), pp. 3111–3119.
26.
BianW., LiS., YangZ., ChenG. and LinZ., A Compare-Aggregate Model with Dynamic-Clip Attention for Answer Selection, (2017), 1987–1990.
27.
YinW., SchützeH., XiangB. and ZhouB., ABCNN: Attention-based convolutional neural network for modeling sentence pairs. arXiv preprint arXiv:1512.05193, (2015).
28.
LiX. and RothD., Learning question classifiers. In Proceedings of the 19th international conference on Computational linguistics-Volume1 (2002), pp. 1–7.
29.
WangY., HuangM., ZhaoL. and ZhuX., Attention-based LSTM for Aspect-level Sentiment Classification, In Proceedings of the 2016 Conference on Empirical Methods in NLP, (2016), pp. 606–615.
30.
ShenY., ZhangQ., ZhangJ., et al., Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding, Information Science and Application (2018), pp. 401–411.
31.
KimY., Convolutional neural networks for sentence classification, In Proceedings of EMNLP, (2014), pp. 1746–1751.
32.
BromleyJ., GuyonI., LecunY., et al., Signature Verification Using a Siamese Time Delay Neural Network. 7th NIPS Conference. (1993).
33.
BertinettoL., Valmadre HenriquesJ., JoãoF., et al., Fully-Convolutional Siamese Networks for Object Tracking. (2016).
34.
RamaV.R., HaloiM., WangG., et al., Gated Siamese Convolutional Neural Network Architecture for Human Reidentification, (2016).
35.
ChenG., ShaoY., TangC., et al., Deep transformation learning for face recognition in the unconstrained scene, Machine Vision and Applications, (2018).
36.
BenajibaY., SunJ., ZhangY., et al., Siamese Networks for Semantic Pattern Similarity, (2018).
37.
PontesE.L., HuetS., LinharesC.A., et al., Predicting the Semantic Textual Similarity with Siamese CNN and LSTM, (2018).
