Skip to main content
SpringerLink
Log in

Multi-lingual handwriting recovery framework based on convolutional denoising autoencoder with attention model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

For several decades, no satisfactory solutions have been provided to the problem of offline handwriting recognition. In the field of online recognition, researchers have had more successful performance, but the ability to extract dynamic information from static images has not been well explored yet. In this paper, we introduce a novel multi-lingual word handwriting recovery framework based on a convolutional denoising autoencoder with an attention model for pen up/down, velocity and temporal order recovery. The proposed framework consists of extracting robust features from a handwriting image using a stacked denoising autoencoder and an encoder Bidirectional Gated Recurrent Unit (BGRU) model. Then, the obtained vectors are decoded to produce an online script with dynamic characteristics using a BGRU with temporal attention. Evaluation is done on a Latin and Arabic Online and offline handwriting character / word databases and the proposed framework achieves high competitive results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Akouaydi H, Njah S, Ouarda W, Samet A, Zaied M, Alimi AM (2019) Convolutional neural networks for online arabic characters recognition with beta-elliptic knowledge domain. In: International Conference on Document Analysis and Recognition Workshops (ICDARW), pp 41–46

  2. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv: 1409.0473

  3. Bhunia AK, Bhowmick A, Bhunia AK, Konwer A, Banerjee P, Roy PP, Pal U (2018) Handwriting trajectory recovery usingend-to-end deep encoder-decoder network. In: 2018 24thInternational Conference on Pattern Recognition (ICPR), pp. 3639–3644

  4. Boubaker H, Elbaati A, Tagougui N, ElAbed H, Kherallah M, Alimi AM (2012) Online Arabic databases and applications. In: Guide to OCR for Arabic Scripts. Springer, London, pp 541–557

    Chapter  Google Scholar 

  5. Chen Z, Yang D, Liang J, Liu X, Wang Y, Peng Z, Huang S (2022) Complex handwriting trajectory recovery: Evaluation metrics and algorithm. In: Proceedings of the Asian Conference on Computer Vision (pp. 1060–1076)

    Google Scholar 

  6. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

  7. Crispo G, Diaz M, Marcelli A, Ferrer MA (2018) Tracking the ballistic trajectory in complex and long handwritten signatures. In: 2018 16th International Conference on Frontiers in Handwriting Recognition(ICFHR), pp 351–356

  8. Dhahri H, Rabhi B, Chelbi S, Almutiry O, Mahmood A, Alimi AM (2021) Automatic detection of COVID-19 using a stacked denoising convolutional autoencoder. Cmc-Comput Mat Contin:3259–3274

  9. Dhieb T, Boubaker H, Ouarda W, Ayed MB, Alimi AM (2019) Deep bidirectional long short-term memory for online arabic writer identification based on beta-elliptic model. In International Conference on Document Analysis and Recognition Workshops (ICDARW), pp 35–40

  10. Diaz M, Ferrer M, Parziale A (2017) Recovering Western On-line Signatures From Image-Based Specimens. Proc. International Conference on Document Analysis and Recognition (ICDAR)

  11. Diaz M, Crispo G, Parziale A, Marcelli A, Ferrer MA (2022) Impact of Writing Order Recovery in Automatic Signature Verification. In: Carmona-Duarte C, Diaz M, Ferrer MA, Morales A (eds) Intertwining Graphonomics with Human Movements. IGS 2022, Lecture Notes in Computer Science, vol 13424. Springer, Cham. https://doi.org/10.1007/978-3-031-19745-1_2

    Chapter  Google Scholar 

  12. Dinh M, Yang HJ, Lee GS, Kim SH, Do LN (2016) Recovery ofdrawing order from multi-stroke English handwritten images based ongraph models and ambiguous zone analysis. Expert Syst Appli 64:352–364

    Article  Google Scholar 

  13. ElBaati A, Alimi AM, Charfi M, Ennaji A (2005) Recovery of temporalinformation from off-line arabic handwritten. In aiccsa, pp. 127-vii

  14. Elbaati A, Kherallah M, Ennaji A, Alimi AM (2009) Temporal order recovery of the scanned handwriting. In: International Conference on Document Analysis and Recognition ICDAR'09, pp. 1116–1120

  15. Hamdi Y, Boubaker H, Dhieb T, Elbaati A, Alimi AM (2019) Hybrid DBLSTM-SVM based Beta-elliptic-CNN Models for Online Arabic Characters Recognition,” In International Conference on Document Analysis and Recognition (ICDAR), pp. 545–550

  16. Hamdi Y, Boubaker H, Alimi AM (2021) Data Augmentation using Geometric, Frequency, and Beta Modeling approaches for Improving Multi-lingual Online Handwriting Recognition. IJDAR 24:283–298. https://doi.org/10.1007/s10032-021-00376-2

    Article  Google Scholar 

  17. Hassaïne A, Al Maadeed S, Bouridane A, (2013) Icdar 2013 competition onhandwriting stroke recovery from offline data. In Document Analysisand Recognition (ICDAR), 12th International Conference pp. 1412–1416

  18. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  19. Hinton E, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. science 313(5786):504–507

    Article  ADS  MathSciNet  CAS  PubMed  Google Scholar 

  20. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput:1735–1780

  21. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  22. Jentzen A, Von Wurstemberger P (2020) Lower error bounds for the stochastic gradient descent optimization algorithm: Sharp convergence rates for slowly and fast decaying learning rates. J Complex 57:101438

    Article  MathSciNet  Google Scholar 

  23. Y. Kato and M. Yasuhara, “Recovery of drawing order from single stroke handwriting images,” IEEE Trans Pattern Anal Mach Intell, vol. 22, no. 9, pp. 938–949, Sep 2000.

  24. Kha VA, Kha HH, Blumenstein M (2016) Extraction of DynamicTrajectory on Multi-Stroke Static Handwriting Images Using LoopAnalysis and Skeletal Graph Model. REV J Electron Commun 6(1–2)

  25. Kherallah M, Elbaati A, Elabed H, Alimi AM (2008) The on/of (LMCA) dual Arabic handwriting database. In: 11th international conference on frontiers in handwriting recognition (ICFHR)

  26. Muramatsu D, Matsumoto T (2007) Effectiveness of pen pressure, azimuth, and altitude features for online signature verification. In: International Conference on Biometrics, pp. 503–512

  27. Nakai M, Sudo T, Shimodaira H, Sagayama S, (2002) Pen pressure features for writer-independent on-line handwriting recognition based on substroke HMM. In: Object recognition supported by user interaction for service robots, pp. 220–223

  28. Noubigh Z, Kherallah M (2017) A survey on handwriting recognition based on the trajectory recovery technique. In: Arabic Script Analysis and Recognition (ASAR), pp. 69–73

  29. Qiao Y, Nishiara M, Yasuhara M (2006) A framework toward restoration of writing order from single-stroked handwriting image. IEEE Trans Pattern Anal Mach Intell 28(11):1724–1737

    Article  PubMed  Google Scholar 

  30. Rabhi B, Elbaati A, Hamdi Y, Alimi AM (2019) Handwriting Recognition Based on Temporal Order Restored by the End-to-End System. Int Conf Doc Analy Recog ICDAR:1231–1236. https://doi.org/10.1109/ICDAR.2019.00199

  31. Rabhi B, Elbaati A, Boubaker H, Alimi AM (2021) Multi-lingual character handwriting framework based on an integrated deep learning based sequence-to-sequence attention model. Memetic Comp 13:459–475. https://doi.org/10.1007/s12293-021-00345-6

    Article  Google Scholar 

  32. Rabhi B, Elbaati A, Hamdani TM, Alimi AM (2021) ASAR 2021 Competition on Online Signal Restoration Using Arabic Handwriting Dhad Dataset. In: Barney Smith E.H., Pal U. (eds) Document Analysis and Recognition – ICDAR 2021 Workshops. ICDAR 2021. Lecture Notes in Computer Science, vol 12916. Springer, Cham. https://doi.org/10.1007/978-3-030-86198-8_26

  33. Rousseau L, Anquetil E, Camillerapp J (2005) Recovery of a drawing orderfrom off-line isolated letters dedicated to on-line recognition. In: Document Analysis and Recognition, 2005. Proceedings. EighthInternational Conference, pp. 1121–1125

  34. Sesa-Nogueras E, Faundez-Zanuy M (2012) Biometric recognition using online uppercase handwritten text. Pattern Recogn, pp. 128–144

  35. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  36. Steinherz T, Doermann D, Rivlin E, Intrator N (2009) Offline loopinvestigation for handwriting analysis. IEEE Trans Patt Analy Mach Intel 31(2):193–209

    Article  Google Scholar 

  37. Vargas JF, Ferrer MA, Travieso CM, Alonso JB (2008) Off-line signature verification based on high pressure polar distribution,” In Procedeeins of the 11th International Conference on Frontiers in Handwriting Recognition, pp. 373–378

  38. Viard-Gaudin C, Lallican PM, Knerr S, Binter P (1999) The ireste on/off (ironoff) dual handwriting database. In: Proceedings of the Fifth International Conference on Document Analysis and Recognition ICDAR, pp. 455–458

  39. Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on Machine learning, pp. 1096–1103

  40. Wang Y, Sonogashira M, Hashimoto A, Iiyama M (2020) Two-stage Fully Convolutional Networks for Stroke Recovery of Handwritten Chinese Character

  41. Yanikoglu B, Kholmatov A (2009) Online signature verification using Fourier descriptors. EURASIP J Adv Sign Proc, pp. 1–13

  42. Zhang R, Chen J, Yang M (2019) Drawing Order Recovery based on deep learning. In: International Conference on Advanced Computational Intelligence (ICACI) (pp. 129–133)

Download references

Acknowledgements

The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES4.

Data Availability 

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Funding

This study was funded by the Ministry of Higher Education and Scientific Research of Tunisia (grant number LR11ES4).

Author information

Authors and Affiliations

  1. University of Sfax, National Engineering School of Sfax, REGIM-Lab.: REsearch Groups in Intelligent Machines, LR11ES48, 3038, Sfax, Tunisia

    Besma Rabhi, Houcine Boubaker & Adel M. Alimi

  2. University of Monastir, Higher Institute of Applied Sciences and Technology of Mahdia, REGIM-Lab.: REsearch Groups in Intelligent Machines, LR11ES48, 5121, Mahdia, Tunisia

    Abdelkarim Elbaati & Houcine Boubaker

  3. Computer Vision and Pattern Recognition Unit, Indian Statistical Institute, 203 B. T. Road, Kolkata, 700108, India

    Houcine Boubaker & Umapada Pal

  4. Department of Electrical and Electronic Engineering Science, Faculty of Engineering and the Built Environment, University of Johannesburg, Johannesburg, South Africa

    Houcine Boubaker & Adel M. Alimi

Authors
  1. Besma Rabhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelkarim Elbaati
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Houcine Boubaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Umapada Pal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Besma Rabhi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rabhi, B., Elbaati, A., Boubaker, H. et al. Multi-lingual handwriting recovery framework based on convolutional denoising autoencoder with attention model. Multimed Tools Appl 83, 22295–22326 (2024). https://doi.org/10.1007/s11042-023-16499-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16499-z

Keywords

Search

Navigation