Skip to main content
SpringerLink
Log in

Handwritten Arabic text recognition using multi-stage sub-core-shape HMMs

  • Special Issue Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

In this paper, we present a multi-stage HMM-based text recognition system for handwritten Arabic. This system employs a novel way of representing Arabic characters by separating the core shapes from the diacritics and then representing these core shapes by smaller units which we term as sub-core shapes. This results in huge reductions in the number of models that need to be trained for the text recognition task. Further, contextual HMM modeling utilizing these sub-core shapes is presented which demonstrates that using sub-core shapes as models improves the contextual HMM system in comparison with a contextual HMM system employing the standard Arabic character shapes as models, and it leads to significantly compact recognizer at the same time. Furthermore, multi-stream contextual sub-core-shape HMMs are presented where the features computed from a sliding window form one stream and its horizontal derivative features are the second stream with each stream having different weights. The system is evaluated on two publicly available databases for different text recognition tasks including conditions where little training data are available. The presented system outperforms the standard character-shape system on all the text recognition tasks on both the databases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Figure adapted from [3]

Fig. 3
Fig. 4

Figure adapted from [2]

Fig. 5
Fig. 6
Fig. 7

Figure adapted from [2]

Fig. 8
Fig. 9

Figure adapted from [2]

Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Abandah, G.A., Jamour, F.T., Qaralleh, E.A.: Recognizing handwritten Arabic words using grapheme segmentation and recurrent neural networks. Int. J. Doc. Anal. Recognit. (IJDAR) 17(3), 275–291 (2014)

    Article  Google Scholar 

  2. Ahmad, I., Fink, G.A.: Multi-stage HMM based Arabic text recognition with rescoring. In: Proceedings of the 13th International Conference on Document Analysis and Recognition (ICDAR 2015), pp. 751–755. IEEE (2015). https://doi.org/10.1109/ICDAR.2015.7333862

  3. Ahmad, I., Rothacker, L., Fink, G., Mahmoud, S.: Novel sub-character HMM models for Arabic text recognition. In: Proceedings of the International Conference on Document Analysis and Recognition. ICDAR (2013). https://doi.org/10.1109/ICDAR.2013.135

  4. Ahmad, I., Fink, G.A., Mahmoud, S.A.: Improvements in sub-character HMM model based Arabic text recognition. In: Proceedings of the 14th International Conference on Frontiers in Handwriting Recognition (ICFHR 2014), pp. 537–542. IEEE, Crete (2014). https://doi.org/10.1109/ICFHR.2014.96

  5. Al-Badr, B., Mahmoud, S.A.: Survey and bibliography of Arabic optical text recognition. Sig. Process. 41(1), 49–77 (1995)

    Article  MATH  Google Scholar 

  6. Al-Hajj Mohamad, R., Likforman-Sulem, L., Mokbel, C.: Combining slanted-frame classifiers for improved HMM-based Arabic handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(7), 1165–1177 (2009)

    Article  Google Scholar 

  7. Almuallim, H., Yamaguchi, S.: A method of recognition of Arabic cursive handwriting. IEEE Trans. Pattern Anal. Mach. Intell. 5, 715–722 (1987)

    Article  Google Scholar 

  8. Azeem, S., Ahmed, H.: Effective technique for the recognition of offline Arabic handwritten words using hidden Markov models. Int. J. Doc. Anal. Recognit. (IJDAR) 16(4), 399–412 (2013). https://doi.org/10.1007/s10032-013-0201-8

    Article  Google Scholar 

  9. BenZeghiba, M.F., Louradour, J., Kermorvant, C.: Hybrid word/part-of-Arabic-word language models for Arabic text document recognition. In: 13th International Conference on Document Analysis and Recognition (ICDAR), 2015, pp. 671–675. IEEE (2015)

  10. Bluche, T., Louradour, J., Knibbe, M., Moysset, B., Benzeghiba, M.F., Kermorvant, C.: The A2iA Arabic handwritten text recognition system at the open HaRT2013 evaluation. In: 11th IAPR International Workshop on Document Analysis Systems (DAS), 2014, pp. 161–165. IEEE (2014)

  11. Cao, H., Natarajan, P., Peng, X., Subramanian, K., Belanger, D., Li, N.: Progress in the Raytheon BBN Arabic offline handwriting recognition system. In: Proceedings of the International Conference on Frontiers in Handwriting Recognition (ICFHR 2014), pp. 555–560. IEEE (2014). https://doi.org/10.1109/ICFHR.2014.99

  12. Chammas, E., Likforman-Sulem, L., Mokbel, C.: Stroke width exploitation to improve automatic recognition of Arabic handwritten texts. In: 2017 1st International Workshop on Arabic Script Analysis and Recognition (ASAR), pp. 74–78. IEEE (2017). https://doi.org/10.1109/ASAR.2017.8067763

  13. Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1923 (1998)

    Article  Google Scholar 

  14. El Abed, H., Märgner, V.: ICDAR 2009-Arabic handwriting recognition competition. Int. J. Doc. Anal. Recognit. (IJDAR) 14(1), 3–13 (2011)

    Article  Google Scholar 

  15. El-Hajj, R., Likforman-Sulem, L., Mokbel, C.: Arabic handwriting recognition using baseline dependant features and hidden Markov modeling. In: Proceedings of Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), pp. 893–897. IEEE (2005)

  16. Ghanmi, N., Awal, A.M., Kooli, N.: Dynamic Bayesian networks for handwritten Arabic word recognition. In: 2017 1st International Workshop on Arabic Script Analysis and Recognition (ASAR), pp. 104–108, IEEE (2017). https://doi.org/10.1109/ASAR.2017.8067769

  17. Giménez, A., Khoury, I., Juan, A.: Windowed Bernoulli mixture HMMs for Arabic handwritten word recognition. In: Proceedings of the 12th International Conference on Frontiers in Handwriting Recognition (ICFHR 2010), pp. 533–538. IEEE (2010). https://doi.org/10.1109/ICFHR.2010.88

  18. Giménez, A., Khoury, I., Andrés-Ferrer, J., Juan, A.: Handwriting word recognition using windowed Bernoulli HMMs. Pattern Recognit. Lett. 35, 149–156 (2014)

    Article  Google Scholar 

  19. Graves, A.: Offline Arabic handwriting recognition with multidimensional recurrent neural networks. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic scripts, pp. 297–313. Springer, Berlin (2012)

  20. Hamdani, M., Doetsch, P., Kozielski, M., Mousa, A.E.D., Ney, H.: The RWTH large vocabulary Arabic handwriting recognition system. In: 11th IAPR international workshop on document analysis systems (DAS), 2014, pp. 111–115. IEEE (2014)

  21. Huang, X., Acero, A., Hon, H.W., Reddy, R.: Spoken language processing: a guide to theory, algorithm, and system development, vol. 1. Prentice Hall PTR, Upper Saddle River (2001)

    Google Scholar 

  22. Jiang, Z., Ding, X., Peng, L., Liu, C.: Analyzing the information entropy of states to optimize the number of states in an HMM-based off-line handwritten Arabic word recognizer. In: 21st International Conference on Pattern Recognition (ICPR), 2012, pp. 697–700. IEEE (2012)

  23. Jiang, Z., Ding, X., Peng, L., Liu, C.: Exploring more representative states of hidden Markov model in optical character recognition: a clustering-based model pre-training approach. Int. J. Pattern Recognit. Artif. Intell. 29(03), 1550014 (2015)

    Article  Google Scholar 

  24. Kessentini, Y., Paquet, T., Hamadou, A.B.: Off-line handwritten word recognition using multi-stream hidden Markov models. Pattern Recognit. Lett. 31(1), 60–70 (2010)

    Article  Google Scholar 

  25. Lorigo, L., Govindaraju, V.: Offline Arabic handwriting recognition: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 712–724 (2006)

    Article  Google Scholar 

  26. Luettin, J., Potamianos, G., Neti, C.: Asynchronous stream modeling for large vocabulary audio-visual speech recognition. In: 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001. Proceedings (ICASSP’01), vol. 1, pp. 169–172. IEEE (2001)

  27. Madi, M.: A study of Arabic letter frequency analysis (2010). http://www.intellaren.com/articles/en/a-study-of-arabic-letter-frequency-analysis

  28. Mahmoud, S.A., Ahmad, I., Alshayeb, M., Al-Khatib, W.G., Parvez, M.T., Fink, G.A., Märgner, V., El Abed, H.: Khatt: Arabic offline handwritten text database. In: 2012 International Conference on Frontiers in Handwriting Recognition, pp. 449–454. IEEE (2012)

  29. Mahmoud, S.A., Ahmad, I., Al-Khatib, W.G., Alshayeb, M., Parvez, M.T., Märgner, V., Fink, G.A.: Khatt: an open arabic offline handwritten text database. Pattern Recognit. 47(3), 1096–1112 (2014)

    Article  Google Scholar 

  30. Manabe, H., Zhang, Z.: Multi-stream HMM for EMG-based speech recognition. In: 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2004. IEMBS’04, vol. 2, pp. 4389–4392. IEEE (2004)

  31. Märgner, V., Abed, H.E.: Arabic handwriting recognition competition. In: Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), vol. 2, pp. 1274–1278. IEEE (2007). https://doi.org/10.1109/ICDAR.2007.4377120

  32. Märgner, V., Abed, H.E.: ICFHR 2010—Arabic handwriting recognition competition. In: Proceedings of the 12th International Conference on Frontiers in Handwriting Recognition (ICFHR 2010), pp. 709–714. IEEE (2010). https://doi.org/10.1109/ICFHR.2010.115

  33. Märgner, V., Abed, H.E.: ICDAR 2011—Arabic handwriting recognition competition. In: Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR 2011), pp. 1444–1448. IEEE (2011) https://doi.org/10.1109/ICDAR.2011.287

  34. Märgner, V., Pechwitz, M., Abed, H.E.: ICDAR 2005 Arabic handwriting recognition competition. In: Proceedings of the Eighth International Conference on Document Analysis and Recognition (ICDAR 2005), vol. 1, pp. 70–74. IEEE (2005). https://doi.org/10.1109/ICDAR.2005.52

  35. Moysset, B., Bluche, T., Knibbe, M., Benzeghiba, M.F., Messina, R., Louradour, J., Kermorvant, C.: The A2ia multi-lingual text recognition system at the second Maurdor evaluation. In: 14th International Conference on Frontiers in Handwriting Recognition (ICFHR), 2014, pp. 297–302. IEEE (2014)

  36. Mozaffari, S., Soltanizadeh, H.: ICDAR2009 handwritten Farsi/Arabic character recognition competition. In: 10th International Conference on Document Analysis and Recognition, 2009. ICDAR’09, pp. 1413–1417. IEEE (2009)

  37. Parvez, M.T., Mahmoud, S.A.: Offline arabic handwritten text recognition: a survey. ACM Comput. Surv. 45(2), 23:1–23:35 (2013). https://doi.org/10.1145/2431211.2431222

    Article  MATH  Google Scholar 

  38. Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H., et al.: IFN/ENIT-database of handwritten Arabic words. In: Proceedings of CIFED, vol. 2, pp. 127–136. Citeseer (2002)

  39. Schambach, M.P., Rottland, J., Alary, T.: How to convert a Latin handwriting recognition system to Arabic. In: Proceedings of the 11th International Conference on Frontiers in Handwriting Recognition (ICFHR 2008), pp. 265–270 (2008)

  40. Stahlberg, F., Vogel, S.: The QCRI recognition system for handwritten Arabic. In: Murino, V., Puppo, E. (eds.) Proceedings of the 18th International Conference on Image Analysis and Processing (ICIAP 2015), pp. 276–286. Springer International Publishing, Genoa (2015)

  41. Wienecke, M., Fink, G.A., Sagerer, G.: Toward automatic video-based whiteboard reading. IJDAR 7(2–3), 188–200 (2005)

    Article  Google Scholar 

  42. Young, S., Evermann, G., Hain, T., Kershaw, D., Moore, G., Odell, J.J., Ollason, D., Povey, D., Valtchev, V., Woodland, P.: The HTK book (for HTK Version 3.2.1). Cambridge University Engineering Department, Cambridge (2002)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank King Fahd University of Petroleum and Minerals (KFUPM) for supporting this work.

Author information

Authors and Affiliations

  1. Information and Computer Science Department, KFUPM, Dhahran, Saudi Arabia

    Irfan Ahmad

  2. Department of Computer Science, TU Dortmund University, Dortmund, Germany

    Gernot A. Fink

Authors
  1. Irfan Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gernot A. Fink
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Irfan Ahmad.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmad, I., Fink, G.A. Handwritten Arabic text recognition using multi-stage sub-core-shape HMMs. IJDAR 22, 329–349 (2019). https://doi.org/10.1007/s10032-019-00339-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-019-00339-8

Keywords

Search

Navigation