Sage Journals: Discover world-class research

Abstract

Spell checking plays an important role in conveying correct information and hence helps in clear communication. Spell checkers for English language are well established. But in case of Indian languages, especially Malayalam lacks a well developed spell checker. The spell checkers that currently exist for Indian languages are based on traditional approaches such as rule based or dictionary based. The rich morphological nature of Malayalam makes spell checking a difficult task. The proposed work is a novel attempt and first of its kind that focuses on implementing a spell checker for Malayalam using deep learning. The spell checker comprises of two processes: error detection and error correction. The error detection section employs a LSTM based neural network which is trained to identify the misspelled words and the position where the error has occurred. The error detection accuracy is measured using the F1 score. Error correction is achieved by the selecting the most probable word from the candidate word suggestions.

Keywords

Malayalam spell checker deep learning natural language processing long short term memory

Get full access to this article

View all access options for this article.

References

Xie

, Avati

, Arivazhagan

, Jurafsky

and Ng

A.Y.

, Neural language correction with character-based attention, arXiv preprint arXiv:1603.09727, 2016.

Brill

and Moore.

R.C.

, An improved error model for noisy channel spelling correction. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pp. 286–293. Association for Computational Linguistics, 2000.

Jain

and Jain

, Detection and correction of non word spelling errors in hindi language. In Data Mining and Intelligent Computing (ICDMIC), 2014 International Conference on, pp. 1–5. IEEE, 2014.

Singh

S.P.

, Kumar

, Singh

, Bhargava

, Goyal

and Sharma

, Frequency based spell checking and rule based grammar checking, In Electrical, Electronics, and Optimization Techniques (ICEEOT), International Conference on, pp. 4435–4439. IEEE, 2016.

Yang

and Xiaobing

, Automatic error detection and correction of text: The state of the art, In Intelligent Networks and Intelligent Systems (ICINIS), 2013 6th International Conference on, pp. 274–277. IEEE, 2013.

Schmidhuber

, Deep learning in neural networks: An overview, Neural Networks61 (2015), 85–117.

LeCun

, Bengio

and Hinton

, Deep learning, Nature521 (2015), 436–444.

Srivastava

and Salakhutdinov

R.R.

, Multimodal learning with deep boltzmann machines, In Advances in neural information processing systems, pp. 2222–2230, 2012.

Gers

F.A.

, Schmidhuber

and Cummins

, Learning to forget: Continual prediction with lstm, Neural Computation12(10) (2000), 2451–2471.

10.

Glorot

, Bordes

and Bengio

, Domain adaptation for large-scale sentiment classification: A deep learning approach, In Proceedings of the 28th international conference on machine learning (ICML-11), pp. 513–520, 2011.

11.

Anand Kumar

A.M.

, Gokul Krishnan

K.S.

, Pooja and Soman

K.P.

, Character based bidirectional lstm for disambiguating tamil part of speech categories, International Journal of Control Theory and Applications10 (2017), 229–235.

12.

Hodge

V.J.

and Austin

, A comparison of standard spell checking algorithms and a novel binary neural approach, Ieee Transactions on Knowledge and Data Engineering15(5) (2003), 1073–1081.

13.

Manjusha

, Anand Kumar

and Soman

K.P.

, Experimental analysis on character recognition using singular value decomposition and random projection, International Journal of Engineering and Technology7(4) (2015).

14.

Sankaran

and Jawahar

C.V.

, Error detection in highly inflectional languages, In Document Analysis and Recognition (ICDAR), 2013 12th International Conference on, IEEE, pp. 1135–1139, 2013.

15.

Gafoor

K.A.

and Remia

K.R.

, Spelling difficulties in Malayalam among lower primary students, Journal of Studies in Teacher Education5(2) (2012), 31–40.

16.

Karpathy

, Johnson

and Fei-Fei

, Visualizing and understanding recurrent networks, arXiv preprint arXiv:1506.02078, 2015.

17.

Graves

, Generating sequences with recurrent neural networks, arXiv preprint arXiv:1308.0850, 2013.

18.

Sutskever

, Martens

and Hinton

G.E.

, Generating text with recurrent neural networks, In Proceedings of the 28th International Conference on Machine Learning (ICML-11) (2011), pp.1017–1024.

Deep learning based spell checker for Malayalam language

Abstract

Keywords

Get full access to this article

References