The paper presents a new corpus for fake news detection in the Urdu language along with the baseline classification and its evaluation. With the escalating use of the Internet worldwide and substantially increasing impact produced by the availability of ambiguous information, the challenge to quickly identify fake news in digital media in various languages becomes more acute. We provide a manually assembled and verified dataset containing 900 news articles, 500 annotated as real and 400, as fake, allowing the investigation of automated fake news detection approaches in Urdu. The news articles in the truthful subset come from legitimate news sources, and their validity has been manually verified. In the fake subset, the known difficulty of finding fake news was solved by hiring professional journalists native in Urdu who were instructed to intentionally write deceptive news articles. The dataset contains 5 different topics: (i) Business, (ii) Health, (iii) Showbiz, (iv) Sports, and (v) Technology. To establish our Urdu dataset as a benchmark, we performed baseline classification. We crafted a variety of text representation feature sets including word n-grams, character n-grams, functional word n-grams, and their combinations. After applying a variety of feature weighting schemes, we ran a series of classifiers on the train-test split. The results show sizable performance gains by AdaBoost classifier with 0.87 F1Fake and 0.90 F1Real. We provide the results evaluated against different metrics for a convenient comparison of future research. The dataset is publicly available for research purposes.
CieriC., MaxwellM., StrasselS.M., TraceyJ., ChoukriK., DeclerckT., GoggiS. and GrobelnikM., Selection Criteria for Low Resource Language Programs, in: Proceedings of the 10th. International Conference on Language Resources and Evaluation, LREC’2016), European Language Resources Association (ELRA), 2016.
2.
PotthastM., KieselJ., ReinartzK., BevendorffJ. and SteinB., A Stylometric Inquiry into Hyperpartisan and Fake News, in: Proceedings of the 56th. Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association forComputationalLinguistics, (2018), pp. 231–240.
3.
ShuK., SlivaA., WangS., TangJ. and LiuH., Fake News Detection on Social Media: A Data Mining Perspective, ACM SIGKDD Explorations Newsletter19(1) (2017), 22–36.
4.
RubinV.L., ChenY. and ConroyN.J., Deception Detection for News: Three Types of Fakes, in: Proceedings of the 78th ASIS&T Annual Meeting: Information Science with Impact: Research in and for the Community, American Society for Information Science, (2015), pp. 83.
5.
ConroyN.J., RubinV.L. and ChenY., Automatic Deception Detection: Methods for Finding fake News, Proceedings of the Association for Information Science and Technology52(1) (2015), 1–4.
6.
NguyenD.M., DoT.H., CalderbankR. and DeligiannisN., Fake News Detection using Deep Markov Random Fields, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), (2019), pp. 1391–1400.
7.
Pérez-RosasV., KleinbergB., LefevreA. and MihalceaR., Automatic Detection of Fake News, in: Proceedings of the 27th International Conference on Computational Linguistics, Association for Computational Linguistics, Santa Fe, New Mexico, USA, (2018), pp. 3391–3401. https://www.aclweb.org/anthology/C18-1287.
ChenY., ConroyN.J. and RubinV.L., Misleading online content: Recognizing clickbait as false news, in: Proceedings of the 2015 ACM on Workshop on Multimodal Deception Detection, ACM, (2015), pp. 15–19.
10.
NazerT.H., XueG., JiY. and LiuH., Intelligent Disaster Response via Social Media Analysis A Survey, ACM SIGKDD Explorations Newsletter19(1) (2017), 46–59.
11.
IvanovV. and TutubalinaE., Clause-Based Approach to Extracting Problem Phrases from User Reviews of Products, (2014), pp. 229–236.
12.
AfrozS., BrennanM. and GreenstadtR., Detecting hoaxes, frauds, and deception in writing style online, in: 2012 IEEE Symposium on Security and Privacy, IEEE, 2012, pp. 461–475.
13.
Posadas-DuránJ.P., Gómez-AdornoH., SidorovG. and Jaime Moreno EscobarJ., Detection of Fake News in a New Corpus for the Spanish Language, Journal of Intelligent & Fuzzy Systems (2018).
14.
BalyR., KaradzhovG., SalehA., GlassJ. and NakovP., Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media, arXiv preprint arXiv:1904.00542 (2019).
15.
FerreiraW. and VlachosA., Emergent: a Novel Data-Set for Stance Classification, in: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL’2016, (2016), pp. 1163–1168.
16.
KrejzlP., HourováB. and SteinbergerJ., Stance Detection in Online Discussions, arXiv preprint arXiv:1701.00504 (2017).
MikolovT., ChenK., CorradoG. and DeanJ., Efficient estimation of word representations in vector space, arXiv preprint arXiv:1301.3781 (2013).
19.
WangW.Y., ” liar, liar pants on fire”: A new benchmark dataset for fake news detection, arXiv preprint arXiv:1705.00648 (2017).
20.
MitraT. and GilbertE., Credbank: A large-scale social media corpus with associated credibility annotations, in: Ninth International AAAI Conference on Web and Social Media, 2015.
21.
ShuK., MahudeswaranD., WangS., LeeD. and LiuH., Fake-NewsNet: A Data Repository with News Content, Social Context and Dynamic Information for Studying Fake News on Social Media, arXiv preprint arXiv:1809.01286 (2018).
22.
PronozaE.V., YagunovaE. and PronozaA., A New Corpus of the Russian Social Network News Feed Paraphrases: Corpus Construction and Linguistic Feature Analysis, in: Advances in Computational Intelligence - 16th Mexican International Conference on Artificial Intelligence, MICAI’2017, (2017), pp. 133–145.
23.
BakerP., HardieA., McEneryT., CunninghamH. and GaizauskasR., EMILLE, A 67 Million Word Corpus of Indic Languages: Data Collection, Mark-up and Harmonisation, in: Proceedings of the 3rd. Language Resources and Evaluation Conference, (2002), pp. 819–825.
24.
SaeedA., NawabR.M.A. and StevensonM., A Word Sense Disambiguation Corpus for Urdu (2018).
25.
MuazA., AliA. and HussainS., Analysis and Development of Urdu POS tagged corpus, in: Proceedings of the 7th. Workshop on Asian Language Resources, IJCNLP, (2009), pp. 24–29.
26.
RazaA.A., HussainS., SarfrazH., UllahI. and SarfrazZ., Design and Development of Phonetically Pich Urdu Speech Corpus, in: 2009 Oriental COCOSDA International Conference on Speech Database and Assessments, IEEE, (2009), pp. 38–43.
27.
WangW.Y., ”Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection, arXiv preprint arXiv:1705.00648 (2017).
28.
Gómez-AdornoH., Posadas-DuranJ.-P., Ríos-ToledoG., SidorovG. and SierraG., Stylometry-based approach for detecting writing style changes in literary texts, Computacíon y Sistemas22(1) (2018), 47–53.
29.
PangB., LeeL. and VaithyanathanS., Thumbs up?: sentiment classification using machine learning techniques, in: Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, Association for Computational Linguistics, (2002), pp. 79–86.
30.
HornR.A. and JohnsonC.R., Norms for vectors and matrices, Ch. 5 in Matrix analysis (1990), 313–386.
31.
LandauerT.K., LSA as a theory of meaning, in: Handbook of latent semantic analysis, Psychology Press, (2007), pp. 15–46.
32.
SidorovG., Miranda-JiménezS., Viveros-JiménezF., GelbukhA., Castro-SánchezN., VelásquezF., Díaz-RangelI., Suárez-GuerraS., TrevinoA. and GordonJ., Empirical Study of Machine Learning based Approach for Opinion Mining in Tweets, in: Mexican international conference on Artificial intelligence, MICAI’2012, Springer, (2012), pp. 1–14.
33.
StamatatosE., A Survey of Modern Authorship Attribution Methods, Journal of the American Society for information Science and Technology60(3) (2009), 538–556.
34.
PangB. and LeeL., Opinion Mining and Sentiment Analysis, Foundations and Trends® in Information Retrieval2(1–2) (2008), 1–135.
35.
BuitinckL., LouppeG., BlondelM., PedregosaF., MuellerA., GriselO., NiculaeV., PrettenhoferP., GramfortA., GroblerJ., LaytonR., VanderPlasJ., JolyA., HoltB. and VaroquauxG., API Design for Machine Learning Software: Experiences from the Scikit-learn Project, in: ECML PKDDWorkshop: Languages for Data Mining and Machine Learning, (2013), pp. 108–122.
