Skip to main content
SpringerLink
Log in

Neural spelling correction: translating incorrect sentences to correct sentences for multimedia

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The aim of a spelling correction task is to detect spelling errors and automatically correct them. In this paper we aim to perform the Korean spelling correction task from a machine translation perspective, allowing it to overcome the limitations of cost, time and data. Based on a sequence to sequence model, the model aligns its source sentence with an ‘error filled sentence’ and its target sentence aligned to the correct counter part. Thus, ‘translating’ the error sentence to a correct sentence. For this research, we have also proposed three new data generation methods allowing the creation of multiple spelling correction parallel corpora from just a single monolingual corpus. Additionally, we discovered that applying the Copy Mechanism not only resolves the problem of overcorrection but even improves it. For this paper, we evaluated our model upon these aspects: Performance comparisons to other models and evaluation on overcorrection. The results show the proposed model to even out-perform other systems currently in commercial use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. https://speller.cs.pusan.ac.kr/

  2. https://www.naver.com/

  3. https://alldic.daum.net/grammar_checkerdo

  4. https://play.google.com/store/apps/details?id=kr.co.systran.eztalky

  5. http://www.llsollu.com/

  6. https://github.com/Parkchanjun/Korea-Spelling-Correction-Resource

References

  1. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473

  2. Byun J, Rim HC, Park SY (2007) Automatic spelling correction rule extraction and application for spoken-style korean text. In: Sixth international conference on advanced language processing and web information technology (ALPIT 2007). IEEE, pp 195–199

  3. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv:1406.1078

  4. Cristo M, Hanada R, Carvalho A, Lores FA, Pimentel MDGC (2017) Fast word recognition for noise channel-based models in scenarios with noise specific domain knowledge. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 607–616

  5. Fivez P, Šuster S, Daelemans W (2017) Unsupervised context-sensitive spelling correction of clinical free-text with word and character n-gram embedding. In: 16th workshop on biomedical natural language processing of the association for computational linguistics, pp 143–148

  6. Gehring J, Auli M, Grangier D, Yarats D, Dauphin YN (2017) Convolutional sequence to sequence learning. In: Proceedings of the 34th international conference on machine learning. JMLR.org, vol 70, pp 1243–1252

  7. Graves A, Mohamed AR, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 6645–6649

  8. Gu J, Lu Z, Li H, Li VO (2016) Incorporating copying mechanism in sequence-to-sequence learning. arXiv:1603.06393

  9. Kim M, Choi SK, Kwon HC (2014) Context-sensitive spelling error correction using inter-word semantic relation analysis. In: 2014 international conference on information science & applications (ICISA). IEEE, pp 1–4

  10. Kim J, Hong T, Kim P (2019) Word2vec based spelling correction method of twitter message. In: Proceedings of the 34th ACM/SIGAPP symposium on applied computing, pp 2016–2019

  11. Klein G, Kim Y, Deng Y, Senellart J, Rush AM (2017) Opennmt: open-source toolkit for neural machine translation. arXiv:1701.02810

  12. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86

    Article  MathSciNet  Google Scholar 

  13. Kwon HC, Kang MY, Choi SJ (2004) Stochastic korean word-spacing with smoothing using korean spelling checker. Int J Comput Process Lang 17 (4):239–252

    Article  Google Scholar 

  14. Lee JH, Kim M, Kwon HC (2017) Improved statistical language model for context-sensitive spelling error candidates. J Korea Multimed Soc 20 (2):371–381

    Article  Google Scholar 

  15. Lee JH, Kim M, Kwon HC (2017) The utilization of local document information to improve statistical context-sensitive spelling error correction. KIISE Trans Comput Pract 23(7):446–451

    Article  Google Scholar 

  16. Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. arXiv:1508.04025

  17. Manohar V, Hadian H, Povey D, Khudanpur S (2018) Semi-supervised training of acoustic models using lattice-free mmi. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 4844–4848

  18. Napoles C, Sakaguchi K, Post M, Tetreault J (2015) Ground truth for grammatical error correction metrics. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 2: short papers) , pp 588–593

  19. Papineni K, Roukos S, Ward T, Zhu WJ (2002) Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, pp 311–318

  20. Park DS, Chan W, Zhang Y, Chiu CC, Zoph B, Cubuk ED, Le QV (2019) Specaugment: a simple data augmentation method for automatic speech recognition. arXiv:1904.08779

  21. Povey D, Ghoshal A, Boulianne G, Burget L, Glembek O, Goel N, Hannemann M, Motlicek P, Qian Y, Schwarz P et al (2011) The kaldi speech recognition toolkit. In: IEEE 2011 workshop on automatic speech recognition and understanding, CONF. IEEE Signal Processing Society

  22. Qin Y, Carlini N, Goodfellow I, Cottrell G, Raffel C (2019) Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. arXiv:1903.10346

  23. Roy S, Ali FB (2019) Unsupervised context-sensitive bangla spelling correction with character n-gram. In: 2019 22nd international conference on computer and information technology (ICCIT). IEEE, pp 1–6

  24. Schabes Y, Roche E (1995) Exact generalization of finite-state transductions: application to grapheme-to-phoneme transcription. In: Submitted to the 23rd meeting of the association for computational linguistics (ACL’95)

  25. Sennrich R, Haddow B, Birch A (2015) Neural machine translation of rare words with subword units. arXiv:1508.07909

  26. Shin G, Seol A, Cho H, Nam K, Pae S (2015) Korean spelling development and linguistic patterns. J Speech Lang Hear Disord 24(2):61–72

    Article  Google Scholar 

  27. Soltau H, Liao H, Sak H (2016) Neural speech recognizer: acoustic-to-word lstm model for large vocabulary speech recognition. arXiv:1610.09975

  28. Sutskever I, Vinyals O, Le Q (2014) Sequence to sequence learning with neural networks. Advances in NIPS

  29. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  30. Yang M (2005) Development of orthographic knowledge among Korean children in grades 1 to 6. University of Virginia

  31. Yujian L, Bo L (2007) A normalized levenshtein distance metric. IEEE Trans Pattern Anal Mach Intell 29(6):1091–1095

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2020-2018-0-01405) supervised by the IITP(Institute for Information & Communications Technology Planning & Evaluation) and National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIP) (No.NRF-2017M3C4A7068189). I am very grateful to my friend Yejin Jang for helping me with correcting English.

Author information

Author notes

    Authors and Affiliations

    1. 311 Aegineung Student Center, College of Informatics, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul, 02841, Korea

      Chanjun Park, Kuekyeng Kim & Heuiseok Lim

    2. Institute of Education, University of Tartu, Tartu, 50103, Estonia

      YeongWook Yang

    3. LLsoLLu, 5, Mabang-ro 10-gil, Seocho-gu, Seoul, Republic of Korea

      Minho Kang

    Authors
    1. Chanjun Park
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Kuekyeng Kim
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. YeongWook Yang
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Minho Kang
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Heuiseok Lim
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Heuiseok Lim.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Chanjun Park and Kuekyeng Kim contributed equally to this work.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Park, C., Kim, K., Yang, Y. et al. Neural spelling correction: translating incorrect sentences to correct sentences for multimedia. Multimed Tools Appl 80, 34591–34608 (2021). https://doi.org/10.1007/s11042-020-09148-2

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-020-09148-2

    Keywords

    Search

    Navigation