Sarcasm detection is becoming increasingly important owing to several reasons such as online brand management, online market research, human-machine interaction and gauging cyberbullying. To improve the performance of sarcasm detection on text, a novel model implementing parallel attention heads based on the concept of transformers at multiple granular levels using a Bidirectional Gated Recurring Unit (Bi-GRU) is proposed. Entropy regularization is implemented to overcome attention collapse and distribute attention mitigating its tendency to over-emphasize particular tokens. Early stopping, and learning rate scheduling used during different implementation and training stages optimize performance and resource utilization. The model is tested on four different sets of embeddings namely: Global Vector (GloVe), FastText, Bidirectional Encoder Representations from Transformers(BERT) and a Latent Semantic Analysis (LSA) reduced ensemble embedding obtained by concatenating the aforementioned embeddings against two datasets. It records maximum accuracy of 93.64% and 81.61% for News Headlines and SARC respectively with FastText outperforming other embeddings. Better results with FastText establish the model's ability to learn well from simpler representations compared to more sophisticated context related patterns established by resource-intensive BERT. The attention heatmaps are plotted to illustrate the interpretability of model. To validate model's effectiveness in real-time, robustness analysis on News Headlines dataset is done using three adversarial testing techniques namely synonym replacement, word dropout and character swap yielding accuracy of 86.12%, 84.16 and 83.21% respectively.
KumarANarapareddyVTSrikanthVA, et al.Sarcasm detection using multi-head attention based bidirectional LSTM. IEEE Access2020; 8: 6388–97.
8.
WangXWangHYangD. Measure and improve robustness in NLP models: A survey. arXiv preprint arXiv:2112.08313. 2021.
9.
GoyalSDoddapaneniSKhapraMMRavindranB. A survey of adversarial defenses and robustness in NLP. ACM Comput Surv2023; 55: 1–39.
10.
RahamanAKuriRIslamS, et al.Sarcasm detection in tweets: a feature-based approach using supervised machine learning models. Int J Adv Comput Sci Appl2021; 12: 6.
11.
KumarASangwanSRAroraA, et al.Sarcasm detection using soft attention-based bidirectional long short-term memory model with convolution network. IEEE Access2019; 7: 23319–28. doi:10.1109/ACCESS.2019.2899260
12.
EkeCINormanAAShuibL. Context-based feature technique for sarcasm identification in benchmark datasets using deep learning and BERT model. IEEE Access2021; 9: 48501–18.
13.
BabanejadNDavoudiHAnAPapagelisM. Affective and contextual embedding for sarcasm detection. Proc 28th Int Conf Comput Linguist2020: 225–43.
14.
PandeyRSinghJP. BERT-LSTM model for sarcasm detection in code-mixed social media post. J Intell Inf Syst2023; 60: 235–54.
15.
KumarASangwanSRSinghAKWadhwaG. Hybrid deep learning model for sarcasm detection in Indian indigenous language using word-emoji embeddings. ACM Trans Asian Low-Resour Lang Inf Process2023; 22: 1–20.
16.
MaheswariSUDhenakaranSS. Improved ensemble based deep learning approach for sarcastic opinion classification. Multimed Tools Appl2023: 1–23.
17.
SharmaDKSinghBAgarwalS, et al.Sarcasm detection over social media platforms using hybrid ensemble model with fuzzy logic. Electronics (Basel)2023; 12: 937.
18.
ThaokarCRoutJKRoutMRayNK. N-Gram based sarcasm detection for news and social media text using hybrid deep learning models. SN Comput Sci2024; 5: 163.
19.
PotamiasRASiolasGStafylopatisAG. A transformer-based approach to irony and sarcasm detection. Neural Comput Appl2020; 32: 17309–20.
20.
MengJZhuYSunSZhaoD. Sarcasm detection based on BERT and attention mechanism. Multimed Tools Appl2023: 1–20.
21.
BăroiuAC. Trăușan-Matu Ș. Comparison of deep learning models for automatic detection of sarcasm context on the MUStARD dataset. Electronics (Basel)2023; 12: 666.
AkulaRGaribayI. Explainable detection of sarcasm in social media. Proc 11th Workshop Comput Appr Subjectivity Sentiment Soc Media Anal2021: 34–9.
24.
KumarADikshitSAlbuquerqueVH. Explainable artificial intelligence for sarcasm detection in dialogues. Wirel Commun Mob Comput2021; 2021: 2939334.
25.
PospelovaNTarasovaASubbotinaN, et al.Explainable artificial intelligence and natural language processing for unraveling deceptive contents. Fusion Pract Appl2024; 14: 146–46.
26.
MajumderNPoriaSPengH, et al.Sentiment and sarcasm classification with multitask learning. IEEE Intell Syst2019; 34: 38–43.
27.
TanYYChowCOKanesanJ, et al.Sentiment analysis and sarcasm detection using deep multi-task learning. Wirel Pers Commun2023; 129: 2213–37.
28.
LiuHWeiRTuG, et al.Sarcasm driven by sentiment: a sentiment-aware hierarchical fusion network for multimodal sarcasm detection. Inf Fusion2024; 108: 102353.
29.
BavkarDKashyapRKhairnarV. Deep hybrid model with trained weights for multimodal sarcasm detection. In: Proceedings of an International Conference on Information, Communication, and Computer Technology, 2023, p.179–94. Springer.
30.
LiYLiYZhangS, et al.An attention-based, context-aware multimodal fusion method for sarcasm detection using inter-modality inconsistency. Knowl Based Syst2024; 287: 111457.
31.
VinothDPrabhavathyP. An intelligent machine learning-based sarcasm detection and classification model on social networks. J Supercomput2022; 78: 10575–94.
32.
KarthikESethukarasiT. Sarcastic user behavior classification and prediction from social media data using firebug swarm optimization-based long short-term memory. J Supercomput2022: 1–25.
33.
AshokDMGhanshyamANSalimSS, et al.Sarcasm detection using genetic optimization on LSTM with CNN. In: Proceedings of an International Conference on Emerging Technologies (INCET), 2020, p.1–4. IEEE.
MisraRGroverJ. Sculpting data for ML: the first act of machine learning. Univ Calif San Diego2021: 158.
36.
KhodakMSaunshiNVodrahalliK. A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579. 2017.
37.
ChandrasekaranGHemanthDJSaravananM. Sarcasm identification in text with deep learning models and Glove word embedding. In: Proc 2nd International Conference Advances in Computing Innovative Technologie Engineering (ICACITE), 2022, p.403–7. IEEE.
38.
OnanA. Topic-enriched word embeddings for sarcasm identification. In: Softw Eng Methods Intell Algorithms. 2019, p. 293–304. Springer.
39.
YacoubADSlimSAboutablA. A survey of sentiment analysis and sarcasm detection: challenges, techniques, and trends. Int J Electr Comput Eng Syst2024; 15: 69–78.
40.
PatilRBoitSGudivadaVNandigamJ. A survey of text representation and embedding techniques in NLP. IEEE Access2023; 11: 36120–46.
NayakDKBollaBK. Efficient deep learning methods for sarcasm detection of news headlines. In: Mach Learn Auton System, 2022, p.371–82. Springer.
43.
MandalPKMahtoR. Deep CNN-LSTM with word embeddings for news headline sarcasm detection. In: International Conference on Information Technology-New Generations (ITNG), 2019, p.495–8. Springer.
44.
LiuLPriestleyJLZhouY, et al.A2text-net: A novel deep neural network for sarcasm detection. In: Proceedings of the International Conference on Cognitive Machine Intelligence (CogMI), 2019, p.118–26. IEEE.
45.
AliRFarhatTAbdullahS, et al.Deep learning for sarcasm identification in news headlines. Appl Sci2023; 13: 5586.
46.
PrasannaMSMShailaSGVadivelA. Polarity classification on twitter data for classifying sarcasm using clause pattern for sentiment analysis. Multimed Tools Appl2023; 82: 32789–825.
47.
LemmensJBurtenshawBLotfiE, et al.Sarcasm detection using an ensemble approach. Proc 2nd Workshop Figurative Lang Process2020: 264–9.
48.
DaduTPantK. Sarcasm detection using context separators in online discourse. Proc 2nd Workshop Figurative Lang Process2020: 51–5.
49.
MehndirattaPSoniD. Identification of sarcasm using word embeddings and hyperparameters tuning. J Discrete Math Sci Cryptogr2019; 22: 465–89.
50.
ShrikhandePSettyVSahaniA. Sarcasm detection in newspaper headlines. In: Proceeding of IEEE international conference on industrial and information systems (ICIIS), 2020, p. 483–7. IEEE.
51.
HazarikaDPoriaSGorantlaS, et al.Cascade: contextual sarcasm detection in online discussion forums. Proc 27th Int Conf Comput Linguist2018: 1837–48.
52.
IlicSMarrese-TaylorEBalazsJAMatsuoY. Deep contextualized word representations for detecting sarcasm and irony. Proc 9th Workshop Comput Appr Subjectivity Sentiment Soc Media Anal2018: 2–7.
53.
DongXLiCChoiJD. Transformer-based context-aware sarcasm detection in conversation threads from social media. arXiv preprint arXiv:2005.11424. 2020.
54.
ShuX. BERT And RoBERTa for sarcasm detection: optimizing performance through advanced fine-tuning. Applied and Computational Engineering2024; 97: 1–11.
55.
SaviniECarageaC. Intermediate-task transfer learning with BERT for sarcasm detection. Mathematics2022; 10: 844.
