Social media platforms allow people across the globe to share their thoughts and opinions and conveniently communicate with each other. Apart from various advantages of social media, it is also misused by a set of users for hate-mongering with toxic and offensive comments. The majority of the earlier proposed toxicity detection methods are primarily focused on the English language, but there is a lack of research on low-resource languages and multilingual text data. We propose an XRBi-GAC framework comprising XLM-RoBERTa, Bi-GRU with self-attention and capsule networks for multilingual toxic text detection. A loss function is also presented, which fuses the binary cross-entropy loss and focal loss to address the class imbalance problem. We evaluated the proposed framework on two datasets, namely, the Jigsaw Multilingual Toxic Comment dataset and HASOC 2019 dataset and achieved F1-score of 0.865 and 0.829, respectively. The results of the experiments show that the proposed framework has outperformed the state-of-the-art multilingual models XLM-RoBERTa and mBERT on both datasets, which shows the versatility and robustness of the proposed XRBi-GAC framework.
ZhaoR., ZhouA. and MaoK., “Automatic detection of cyberbullying on social networks based on bullying features,” in Proceedings of the 17th international conference on distributed computing and networking, pp. 1–6, 2016.
4.
PranckeviciusT. and MarcinkevičiusV., “Application of logistic regression with part-of-the-speech tagging for multi-class text classification,” in 2016 IEEE 4th workshop on advances in information, electronic and electrical engineering (AIEEE), pp. 1–5, IEEE, 2016.
5.
GaydhaniA., DomaV., KendreS. and BhagwatL., “Detecting hate speech and offensive language on twitter using machine learning: An n-gram and tfidf based approach,” arXiv preprint arXiv:1809.08651, 2018.
6.
NeogiA.S., GargK.A., MishraR.K. and DwivediY.K., Sentiment analysis and classification of indian farmers’ protest using twitter data, International Journal of Information Management DataInsights1(2) (2021), 100019.
7.
VaswaniA., ShazeerN., ParmarN., UszkoreitJ., JonesL., GomezA.N., KaiserŁ. and PolosukhinI., Attention is all you need, Advances in Neural Information Processing systems30(2017).
8.
DevlinJ., ChangM.-W., LeeK. and ToutanovaK., “Bert: Pretraining of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
9.
SanhV., DebutL., ChaumondJ. and WolfT., “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” arXiv preprint arXiv:1910.01108, 2019.
10.
LiuY., OttM., GoyalN., DuJ., JoshiM., ChenD., LevyO., ZettlemoyerM. L. and StoyanovV., “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019.
11.
LanZ., ChenM., GoodmanS., GimpelK., SharmaP. and SoricutR., “Albert: A lite bert for self-supervised learning of language representations,” arXiv preprint arXiv:1909.11942, 2019.
12.
GaoL. and HuangR., “Detecting online hate speech using context aware models,” arXiv preprint arXiv:1710.07395, 2017.
13.
SinghP. and ChandS., “Pardeep at semeval-2019 task 6: Identifying and categorizing offensive language in social media using deep learning,” in Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 727–734, 2019.
14.
IbrahimM., TorkiM. and El-MakkyN., “Imbalanced toxic comments classification using data augmentation and deep learning,” in 2018 17th IEEE international conference on machine learning and applications (ICMLA), pp. 875–878, IEEE, 2018.
15.
Van AkenB., RischJ., KrestelR. and LoserA., “Challenges for toxic comment classification: An in-depth error analysis,” arXiv preprint arXiv:1809.07572, 2018.
16.
RanasingheT., ZampieriM. and HettiarachchiH., “Brums at hasoc 2019: Deep learning models for multilingual hate speech and offensive language identification.,” in FIRE (working notes), pp. 199–207, 2019.
17.
LampleG. and ConneauA., “Cross-lingual language model pretraining,” arXiv preprint arXiv:1901.07291, 2019.
18.
DongX.L. and de MeloG., “A robust self-learning framework for cross-lingual text classification,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pp. 6306–6310, 2019.
19.
WangS., LiuJ., OuyangX. and SunY., “Galileo at semeval- 2020 task 12: Multi-lingual learning for offensive language identification using pre-trained language models,” arXiv preprint arXiv:2010.03542, 2020.
20.
SaiS. and SharmaY., “Siva@ hasoc-dravidian-codemixfire- 2020: Multilingual offensive speech detection in codemixed and romanized text.,” in FIRE (Working Notes), pp. 336–343, 2020.
21.
PantK. and DaduT., “Cross-lingual inductive transfer to detect offensive language,” arXiv preprint arXiv:2007.03771, 2020.
22.
MalikP., AggrawalA. and VishwakarmaD.K., “Toxic speech detection using traditional machine learning models and bert and fasttext embedding with deep neural networks,” in 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), pp. 1254–1259, IEEE, 2021.
23.
SongG., HuangD. and XiaoZ., “A study of multilingual toxic textdetection approaches under imbalanced sample distribution,”, Information12(5) (2021), 205.
24.
JhaveriM., RamaiyaD. and ChadhaH.S., “Toxicity detection for indic multilingual social media content,” arXiv preprint arXiv:2201.00598, 2022.
25.
LashkarashviliN. and TsintsadzeM., “Toxicity detection in onlinegeorgian discussions,”, International Journal of InformationManagement Data Insights2(1) (2022), 100062.
26.
MandlT., ModhaS., MajumderP., PatelD., DaveM., MandliaC. and PatelA., “Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indoeuropean languages,” in Proceedings of the 11th forum for information retrieval evaluation, pp. 14–17, 2019.
27.
RumelhartD.E., HintonG.E. and WilliamsR.J., Learningrepresentations by back-propagating errors, Nature323(6088) (1986), 533–536.
28.
HochreiterS. and SchmidhuberJ., Long short-term memory, Neural Computation9(8) (1997), 1735–1780.
29.
ChungJ., GulcehreC., ChoK. and BengioY., “Empirical evaluation of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555, 2014.
30.
SchusterM. and PaliwalK.K., Bidirectional recurrent neuralnetworks, IEEE Transactions on Signal Processing45(11) (1997), 2673–2681.
31.
SabourS., FrosstN. and HintonG.E., Dynamic routing betweencapsules, Advances in Neural Information Processing Systems30 (2017).
32.
SaifM.A., MedvedevA.N., MedvedevM.A. and AtanasovaT., “Classification of online toxic comments using the logisticregression and neural networks models,”, in AIP conference proceedings2048 (2018), 060011, AIP Publishing LLC.
33.
DhamijaT., KataryaR., et al., “Comparative analysis of machine learning and deep learning algorithms for detection of online hate speech,” in Advances in Mechanical Engineering, pp. 509–520, Springer, 2021.
34.
GhoshS., KumarS., LepchaS. and JainS.S., “Toxic text classification,” in Data Science and Security, pp. 251–260, Springer, 2021.
