Skip to main content
SpringerLink
Log in

Towards Boosting the Accuracy of Non-latin Scene Text Recognition

  • Conference paper
  • First Online:
Document Analysis and Recognition – ICDAR 2021 Workshops (ICDAR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12916))

Included in the following conference series:

Abstract

Scene-text recognition is remarkably better in Latin languages than the non-Latin languages due to several factors like multiple fonts, simplistic vocabulary statistics, updated data generation tools, and writing systems. This paper examines the possible reasons for low accuracy by comparing English datasets with non-Latin languages. We compare various features like the size (width and height) of the word images and word length statistics. Over the last decade, generating synthetic datasets with powerful deep learning techniques has tremendously improved scene-text recognition. Several controlled experiments are performed on English, by varying the number of (i) fonts to create the synthetic data and (ii) created word images. We discover that these factors are critical for the scene-text recognition systems. The English synthetic datasets utilize over 1400 fonts while Arabic and other non-Latin datasets utilize less than 100 fonts for data generation. Since some of these languages are a part of different regions, we garner additional fonts through a region-based search to improve the scene-text recognition models in Arabic and Devanagari. We improve the Word Recognition Rates (WRRs) on Arabic MLT-17 and MLT-19 datasets by \(24.54\%\) and \(2.32\%\) compared to previous works or baselines. We achieve WRR gains of \(7.88\%\) and \(3.72\%\) for IIIT-ILST and MLT-19 Devanagari datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We also investigated other reasons for low recognition rates in non-Latin languages, like comparing the size of word images of Latin and non-Latin real datasets but could not find any significant variations (or exciting differences). Moreover, we observe very high word recognition rates (\(>90\%\)) when we tested our non-Latin models on the held-out synthetic datasets, which shows that learning to read the non-Latin glyphs is trivial for the existing deep models. Refer https://github.com/firesans/STRforIndicLanguages for more details.

  2. 2.

    https://catalist-2021.github.io/.

  3. 3.

    Additional fonts we found using region-based online search are available at: www.sanskritdocuments.org/, www.tinyurl.com/n84kspbx, www.tinyurl.com/7uz2fknu, www.ctan.org/tex-archive/fonts/shobhika?lang=en, www.hindi-fonts.com/, www.fontsc.com/font/tag/arabic, more fonts are shared on https://github.com/firesans/NonLatinPhotoOCR.

  4. 4.

    We could not obtain the ARASTEC dataset we discussed in the previous section.

References

  1. Bušta, M., Neumann, L., Matas, J.: Deep TextSpotter: an end-to-end trainable scene text localization and recognition framework. In: ICCV (2017)

    Google Scholar 

  2. Bušta, M., Patel, Y., Matas, J.: E2E-MLT - an unconstrained end-to-end method for multi-language scene text. In: Carneiro, G., You, S. (eds.) ACCV 2018. LNCS, vol. 11367, pp. 127–143. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21074-8_11

    Chapter  Google Scholar 

  3. Chng, C.K., Chan, C.S.: Total-text: a comprehensive dataset for scene text detection and recognition. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 935–942 (2017)

    Google Scholar 

  4. Dwivedi, A., Saluja, R., Kiran Sarvadevabhatla, R.: An OCR for classical Indic documents containing arbitrarily long words. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020

    Google Scholar 

  5. Gupta, A., Vedaldi, A., Zisserman, A.: Synthetic data for text localisation in natural images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2315–2324 (2016)

    Google Scholar 

  6. Huang, Z., Zhong, Z., Sun, L., Huo, Q.: Mask R-CNN with pyramid attention network for scene text detection. In: WACV, pp. 764–772. IEEE (2019)

    Google Scholar 

  7. Iwamura, M., Matsuda, T., Morimoto, N., Sato, H., Ikeda, Y., Kise, K.: Downtown Osaka scene text dataset. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 440–455. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_32

    Chapter  Google Scholar 

  8. Jaderberg, M., Simonyan, K., Vedaldi, A., Zisserman, A.: Synthetic data and artificial neural networks for natural scene text recognition. In: Workshop on Deep Learning, NIPS (2014)

    Google Scholar 

  9. Jung, J., Lee, S., Cho, M.S., Kim, J.H.: Touch TT: scene text extractor using touchscreen interface. ETRI J. 33(1), 78–88 (2011)

    Article  Google Scholar 

  10. Karatzas, D., et al.: ICDAR 2013 robust reading competition. ICDAR 2013, pp. 1484–1493. IEEE Computer Society, USA (2013)

    Google Scholar 

  11. Liu, W., Chen, C., Wong, K.Y.K., Su, Z., Han, J.: STAR-Net: a SpaTial attention residue network for scene text recognition. In: BMVC, vol. 2 (2016)

    Google Scholar 

  12. Mathew, M., Jain, M., Jawahar, C.: Benchmarking scene text recognition in Devanagari, Telugu and Malayalam. In: ICDAR, vol. 7, pp. 42–46. IEEE (2017)

    Google Scholar 

  13. Mishra, A., Alahari, K., Jawahar, C.V.: Scene text recognition using higher order language priors. In: BMVC (2012)

    Google Scholar 

  14. Nayef, N., et al.: ICDAR2019 robust reading challenge on multi-lingual scene text detection and recognition-RRC-MLT-2019. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1582–1587. IEEE (2019)

    Google Scholar 

  15. Nayef, N., et al.: Robust reading challenge on multi-lingual scene text detection and script identification - RRC-MLT. In: 14th ICDAR, vol. 1, pp. 1454–1459. IEEE (2017)

    Google Scholar 

  16. Phan, T., Shivakumara, P., Tian, S., Tan, C.: Recognizing text with perspective distortion in natural scenes. In: 2013 IEEE International Conference on Computer Vision, pp. 569–576 (2013)

    Google Scholar 

  17. Risnumawan, A., Shivakumara, P., Chan, C.S., Tan, C.L.: A robust arbitrary text detection system for natural scene images. Expert Syst. Appl. 41, 8027–8048 (2014)

    Article  Google Scholar 

  18. Saluja, R., Adiga, D., Chaudhuri, P., Ramakrishnan, G., Carman, M.: Error detection and corrections in Indic OCR using LSTMs. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 17–22. IEEE (2017)

    Google Scholar 

  19. Saluja, R., Maheshwari, A., Ramakrishnan, G., Chaudhuri, P., Carman, M.: OCR On-the-Go: robust end-to-end systems for reading license plates and street signs. In: 15th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 154–159. IEEE (2019)

    Google Scholar 

  20. Shahab, A., Shafait, F., Dengel, A.: ICDAR 2011 robust reading competition challenge 2: reading text in scene images. In: 2011 International Conference on Document Analysis and Recognition, pp. 1491–1496 (2011)

    Google Scholar 

  21. Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2016)

    Article  Google Scholar 

  22. Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., Bai, X.: ASTER: an attentional scene text recognizer with flexible rectification. IEEE Trans. Pattern Anal. Mach. Intell. 41(9), 2035–2048 (2018)

    Article  Google Scholar 

  23. Shi, B., et al.: ICDAR2017 competition on reading Chinese text in the wild (RCTW-17). In: 14th ICDAR, vol. 1, pp. 1429–1434. IEEE (2017)

    Google Scholar 

  24. Sun, Y., et al.: ICDAR 2019 competition on large-scale street view text with partial labeling - RRC-LSVT. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1557–1562. IEEE (2019)

    Google Scholar 

  25. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, Inception-Resnet and the impact of residual connections on learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)

    Google Scholar 

  26. Tounsi, M., Moalla, I., Alimi, A.M., Lebouregois, F.: Arabic characters recognition in natural scenes using sparse coding for feature representations. In: 13th ICDAR, pp. 1036–1040. IEEE (2015)

    Google Scholar 

  27. Veit, A., Matera, T., Neumann, L., Matas, J., Belongie, S.: COCO-Text: dataset and benchmark for text detection and recognition in natural images. arXiv preprint arXiv:1601.07140 (2016)

  28. Wang, K., Babenko, B., Belongie, S.: End-to-end scene text recognition. In: 2011 International Conference on Computer Vision, pp. 1457–1464 (2011)

    Google Scholar 

  29. Wang, K., Babenko, B., Belongie, S.: End-to-end scene text recognition. In: ICCV, pp. 1457–1464. IEEE (2011)

    Google Scholar 

  30. Wang, K., Belongie, S.: Word spotting in the wild. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 591–604. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_43

    Chapter  Google Scholar 

  31. Yuan, T., Zhu, Z., Xu, K., Li, C., Mu, T., Hu, S.: A large Chinese text dataset in the wild. J. Comput. Sci. Technol. 34(3), 509–521 (2019). https://doi.org/10.1007/s11390-019-1923-y

    Article  Google Scholar 

  32. Zeiler, M.: Adadelta: An adaptive learning rate method 1212, December 2012

    Google Scholar 

  33. Zhang, R., et al.: ICDAR 2019 robust reading challenge on reading Chinese text on signboard. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1577–1581. IEEE (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Vision Information Technology, International Institute of Information Technology, Hyderabad, 500032, India

    Sanjana Gunna, Rohit Saluja & C. V. Jawahar

Authors
  1. Sanjana Gunna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rohit Saluja
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. V. Jawahar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sanjana Gunna .

Editor information

Editors and Affiliations

  1. Boise State University, Boise, ID, USA

    Elisa H. Barney Smith

  2. Indian Statistical Institute, Kolkata, India

    Umapada Pal

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gunna, S., Saluja, R., Jawahar, C.V. (2021). Towards Boosting the Accuracy of Non-latin Scene Text Recognition. In: Barney Smith, E.H., Pal, U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science(), vol 12916. Springer, Cham. https://doi.org/10.1007/978-3-030-86198-8_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86198-8_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86197-1

  • Online ISBN: 978-3-030-86198-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation