Skip to main content
SpringerLink
Log in

Chinese Wrong Words Detection and Correction Algorithm Based on Sentence Perplexity and Iterative Correction

  • Conference paper
  • First Online:
Cyber Security Intelligence and Analytics (CSIA 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1147))

  • 1285 Accesses

Abstract

With the rapid development of the field of artificial intelligence, the traditional way to check whether there is a typo in the text is very inefficient. People have been looking for a way to automatically detect and correct wrong words used in texts. The use of artificial intelligence technology for detection and correction of typography came into being. The defects and causes of the existing Chinese wrong words detection and correction technology is analyzed in this paper. The algorithm is designed to detect and correct the wrong words twice and judge the interval between the first result and the second result. The sentence perplexity is calculated by combining the deep neural network language model, forming a double filter to reduce the misjudgment in the process of wrong words detection. Experiments in various text styles indicate that the improved algorithm can reduce more than 90% of misjudgments in the process of detection and correction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Julie. Automatic detection of Chinese typography. Sichuan Foreign Studies University (2016). (in Chinese)

    Google Scholar 

  2. Xu, M.: Research and implementation of automatic detection and correction of typography in online business user reviews. Beijing University of Technology (2016). (in Chinese)

    Google Scholar 

  3. https://my.oschina.net/airship/blog/3014199

  4. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)

    MATH  Google Scholar 

  5. https://www.jianshu.com/p/3f22fc76a9e5

  6. Li, F., Ke, J.: Research progress of word vector semantic representation. Inf. Sci. 37(05), 155–165 (2019). (in Chinese)

    Google Scholar 

  7. Li, H., Zhang, H.: Book review emotion analysis method based on word vector and CNN. J. Test. Technol. (2019). (in Chinese)

    Google Scholar 

  8. Mikolov, T.: Statistical Language Models Based on Neural Networks. Brno University of Technology (2012)

    Google Scholar 

  9. Mikolov, T., Yih, W.T., Zweig, G.: Linguistic regularities continuous space word representations. Assoc. Comput. Linguist. (2013)

    Google Scholar 

  10. https://github.com/shibing624/pycorrector

Download references

Acknowledgments

This work was partially supported by the following research grants: (1) No. 2016NXY46 from the Research Foundation of Nanjing Xiaozhuang University and (2) No. 19AIP07 from the Research Foundation of the Key Laboratory of Intelligent Information Processing of Nanjing Xiaozhuang University.

Author information

Authors and Affiliations

  1. Information Engineering School, Nanjing Xiaozhuang University, Nanjing, 211171, China

    Wanli Song & Chunxu Zhu

  2. School of Computer and Information Engineering, Hohai University, Nanjing, 210098, China

    Wanli Song

  3. Key Laboratory of Intelligent Information Processing, Nanjing Xiaozhuang University, Nanjing, 211171, China

    Wanli Song

Authors
  1. Wanli Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunxu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanli Song .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Zheng Xu

  2. Department of Software Engineering and Game Development, Kennesaw State University, Marietta, GA, USA

    Reza M. Parizi

  3. Manchester Metropolitan University, Stockport, UK

    Mohammad Hammoudeh

  4. School of Engineering and Sciences, Tecnologico de Monterrey, Puebla, Mexico

    Octavio Loyola-González

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Song, W., Zhu, C. (2020). Chinese Wrong Words Detection and Correction Algorithm Based on Sentence Perplexity and Iterative Correction. In: Xu, Z., Parizi, R., Hammoudeh, M., Loyola-González, O. (eds) Cyber Security Intelligence and Analytics. CSIA 2020. Advances in Intelligent Systems and Computing, vol 1147. Springer, Cham. https://doi.org/10.1007/978-3-030-43309-3_66

Download citation

Publish with us

Policies and ethics

Search

Navigation