Aspect-based sentiment analysis (ABSA) is a fine-grained analysis of sentiments and opinions expressed in text document which set aside the naïve assumption that a document expresses opinion about a single topic or aspect. The main objective of ABSA is to spot, extract and identify the polarity of different entities and aspects in an opinionated document. Based on the previous works, ABSA can be categorized into three subtasks: Aspect-category sentiment analysis (ACSA), Opinion Target Expression Sentiment Analysis (OTESA) and Aspect-term sentiment analysis (ATSA). This research presents an end-to-end multi-task approach to performing the three categories of ABSA on a single pipeline. A ternary multitask learning objectives classifiers were built on top of the baseline spanBERT language model which was originally pretrained for span extraction. The input to the model consists of two merged segments of entity premises and context data hypothesis in a similar passion to reading comprehension downstream task in natural language processing. The ternary downstream tasks were built on the contextualized output embeddings of pretrained spanBERT entangled with cross-layer attention mechanism to associate context with the aspect-term span extraction, aspect sentiment polarity detection and entity-aspect entailment. A span masking approach was also proposed to address multiple-aspects text using an iterative outputs-inputs loopback. The span masking process replaces each word in a previously detected span of text with a special [MASK] character and then feeds back the entire sentence into the encoder input of the model for next run. The technique forces the encoder to look elsewhere for the next span prediction. The loopback span masking terminates when the span classifiers predict a special token [CLS] as the beginning and end of the span signaling the absent of relatable span to be extracted. Experimental results validate the approach as impressive results were obtained outperforming most of the compared research with benchmark ABSA datasets.
AshirA. M.TahaM. A. (2024). BERT with an augmented cross-attention decoder (BERT-ACD) for binary and fine-grained multiband sentiment detection. Intelligenza Artificiale, 19(1), 3–16. https://doi.org/10.1177/17248035241291687
2.
AugustyniakŁKajdanowiczT.KazienkoP. (2021). Comprehensive analysis of aspect term extraction methods using various text embeddings. Comput Speech Lang, 69(1). https://doi.org/10.1016/j.csl.2021.101217
3.
BahdanauD.ChoK.BengioY. (2016). Neural Machine Translation by Jointly Learning to Align and Translate. CoRR, abs/1409.0473.
4.
BollegalaD.WeirD.CarrollJ. (2013). Cross-domain sentiment classification using a sentiment sensitive thesaurus. IEEE Trans Knowl Data Eng, 25(8), 1719–1731. https://doi.org/10.1109/TKDE.2012.103
5.
BordoloiM.BiswasS. K. (2023). Sentiment analysis: A survey on design framework, applications and future scopes. Artif Intell Rev. https://doi.org/10.1007/s10462-023-10442-2
ChenP.SunZ.BingL.YangW. (2017). Recurrent attention network on memory for aspect sentiment analysis. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (pp. 452–461). Association for Computational Linguistics. https://doi.org/10.18653/v1/D17-1047
8.
DevlinJ.ChangM.-W.LeeK.GoogleK. T.LanguageA. I. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.18653/v1/N19-1423
9.
GaoY.GlowackaD. (2016). Deep gate recurrent neural network. In DurrantR. J.KimK.-E. (Eds.), Proceedings of the 8th Asian conference on machine learning, Proceedings of Machine Learning Research, vol. 63. The University of Waikato (pp. 350–365). PMLR. [Online]. Available: https://proceedings.mlr.press/v63/gao30.html
HeR.LeeW. S.NgH. T.DahlmeierD. (2018). Exploiting document knowledge for aspect-level sentiment classification. In GurevychI.MiyaoY. (Eds.), Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: Short papers) (pp. 579–585). Association for Computational Linguistics. https://doi.org/10.18653/v1/P18-2092
12.
HeR.LeeW. S.NgH. T.DahlmeierD. (2019). An interactive multi-task learning network for End-to-End aspect-based sentiment analysis. In KorhonenA.TraumD.MàrquezL. (Eds.), Proceedings of the 57th annual meeting of the association for computational linguistics (pp. 504–515). Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1048
HuM.PengY.HuangZ.LiD.LvY. (2019). Open-domain targeted sentiment analysis via span-based extraction and classification. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 537–546). Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1051
15.
JoshiM.ChenD.LiuY.WeldD. S.ZettlemoyerL.LevyO. (2020). SpanBERT: Improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist, 8, 64–77. https://doi.org/10.1162/tacl_a_00300
LiX.BingL.LiP.LamW. (2019a). A unified model for opinion target extraction and target sentiment prediction. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, in AAAI’19/IAAI’19/EAAI’19. AAAI Press. https://doi.org/10.1609/aaai.v33i01.33016714
18.
LiX.BingL.LiP.LamW. (2019b). A unified model for opinion target extraction and target sentiment prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6714–6721. https://doi.org/10.1609/aaai.v33i01.33016714
19.
LiX.BingL.ZhangW.LamW. (2019c). Exploiting BERT for end-to-end aspect-based sentiment analysis. Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019) (pp. 34–41). Association for Computational Linguistics. https://doi.org/10.18653/v1/D19-5505
LiH.LuW. (2017). Learning latent sentiment scopes for entity-level sentiment analysis. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, in AAAI’17 (pp. 3482–3489). AAAI Press.
22.
LiJ.XiaoL. (2020). Syrapropa at SemEval-2020 task 11: BERT-based models design for propagandistic technique and span detection. Proceedings of the Fourteenth Workshop on Semantic Evaluation (pp. 1808–1816). International Committee for Computational Linguistics. https://doi.org/10.18653/v1/2020.semeval-1.237
LiangY.MengF.ZhangJ.XuJ.ChenY.ZhouJ. (2020) A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis.
25.
LiuS.LiW.WuY.SuQ.SunX. (2020). Jointly modeling aspect and sentiment with dynamic heterogeneous graph neural networks.
26.
LiuB.ZhangL. (2012). A survey of opinion mining and sentiment analysis. In AggarwalC.ZhaiC. (Ed.), Mining text data (pp. 415–463). Springer US. https://doi.org/10.1007/978-1-4614-3223-4_13
27.
MaoY.ShenY.YangJ.ZhuX.CaiL. (2022). Seq2Path: Generating sentiment tuples as paths of a tree. In MuresanS.NakovP.VillavicencioA. (Eds.), Findings of the association for computational linguistics: ACL 2022 (pp. 2215–2225). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.findings-acl.174
28.
NguyenH.NguyenM. L. (2018). A deep neural architecture for sentence-level sentiment classification in twitter social networking. In HasidaK.PaW. (Eds.), Computational linguistics (pp. 15–27). Springer Singapore.
29.
PangB.LeeL.VaithyanathanS. (2002). Thumbs up? Sentiment classification using machine learning techniques. Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002) (pp. 79–86). Association for Computational Linguistics. https://doi.org/10.3115/1118693.1118704
30.
PetersM. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyeret, L. (2018). Deep contextualized word representations. https://doi.org/10.48550/arXiv.1802.05365
31.
PontikiM., et al. (2016). SemEval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016) (pp. 19–30). Association for Computational Linguistics. https://doi.org/10.18653/v1/S16-1002
RajpurkarP.ZhangJ.LopyrevK.LiangP. (2016). SQuAD: 100,000+ questions for machine comprehension of text. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 2383–2392). Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1264
34.
ReiM. (2017). Semi-supervised multitask learning for sequence labeling. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2121–2130). Association for Computational Linguistics. https://doi.org/10.18653/v1/P17-1194
35.
SeoM. J.KembhaviA.FarhadiA.HajishirziH. (2017). Bidirectional attention flow for machine comprehension. 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net. [Online]. Available: https://openreview.net/forum?id=HJ0UKP9ge
36.
SøgaardA.GoldbergY. (2016). Deep multi-task learning with low level tasks supervised at lower layers. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 231–235). Association for Computational Linguistics. https://doi.org/10.18653/v1/P16-2038
37.
TangD.QinB.FengX.LiuT. (2016). Effective LSTMs for target-dependent sentiment classification. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp. 3298–3307). The COLING 2016 Organizing Committee. [Online]. Available: https://aclanthology.org/C16-1311
WangY.HuangM.ZhuX.ZhaoL. (2016). Attention-based LSTM for aspect-level sentiment classification. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 606–615). Association for Computational Linguistics. https://doi.org/10.18653/v1/D16-1058
40.
WangS.JiangJ. (2017). Machine comprehension using match-LSTM and answer pointer. 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net. [Online]. Available: https://openreview.net/forum?id=B1-q5Pqxl
WilsonT.WiebeJ.HoffmannP. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In MooneyR.BrewC.ChienL.-F.KirchhoffK. (Eds.), Proceedings of human language technology conference and conference on empirical methods in natural language processing (pp. 347–354). Association for Computational Linguistics. [Online]. Available: https://aclanthology.org/H05-1044
43.
XuH.LiuB.ShuL.YuP. S. (2018) Double embeddings and CNN-based sequence labeling for aspect extraction. In GurevychI.MiyaoY. (Eds.), Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: Short papers) (pp. 592–598). Association for Computational Linguistics. https://doi.org/10.18653/v1/P18-2094
44.
YangZ.DaiZ.YangY.CarbonellJ.SalakhutdinovR.LeQ. V. (2019). XLNet: Generalized autoregressive pretraining for language understanding. Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc..
45.
YangZ.SalakhutdinovR.CohenW. W. (2017). Transfer learning for sequence tagging with hierarchical recurrent networks.
46.
ZhangW.FengY.MengF.YouD.LiuQ. (2019). Bridging the gap between training and inference for neural machine translation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 4334–4343). Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1426
47.
ZhangW.LiX.DengY.BingL.LamW. (2023). A survey on aspect-based sentiment analysis: Tasks, methods, and challenges. IEEE Trans Knowl Data Eng, 35(11), 11019–11038. https://doi.org/10.1109/TKDE.2022.3230975
48.
ZhangM.ZhangY.VoD.-T. (2016). Gated neural networks for targeted sentiment analysis. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, in AAAI’16 (pp. 3087–3093). AAAI Press.
