Skip to main content
SpringerLink
Log in

Ethiopian sign language recognition using deep convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, several technologies have been utilized to bridge the communication gap between persons who have hearing or speaking impairments and those who don't. This paper presents the development of a novel sign language recognition system which translates Ethiopian sign language (ETHSL) to Amharic alphabets using computer vision technology and Deep Convolutional Neural Network (CNN). The system accepts sign language images as input and gives Amharic text as the desired output. The proposed system comprises of three main stages which are: preprocessing, feature extraction, and recognition. The methodology employed involves data acquisition, preprocessing the acquired data, background normalization, image resizing, region of interest (ROI) identification, noise removal, brightness adjustment, and feature extraction, while Deep Convolutional Neural Network (CNN) was used for end-to-end classification. The data used in this study was acquired from students with hearing impairments at the Debre Markos Teaching College with an iPhone 6s phone which has a resolution of 3024 × 4020. The images are in JPEG file format and were collected in a controlled environment. The proposed system was implemented using Kera’s (Tensorflow2.3.0 as backend) in python and tested using the image dataset collected from Debre Markos Teaching College graduating students of 2012. The results show that the running time was minimized by adjusting the images to a suitable size and color. In addition, the results show an improved recognition accuracy compared to previous works. The proposed model achieves 98.5% training, 95.59% validation, and 98.3% testing accuracy of recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

References

  1. Tamiru NK, Tekeba M, Salau AO (2021) Recognition of Amharic sign language with Amharic alphabet signs using ANN and SVM. Vis Comput 0123456789. https://doi.org/10.1007/s00371-021-02099-1

  2. Bantupalli K (2018) American sign language recognition using deep learning and computer vision. IEEE Int Conf Big Data (Big Data), 4896–4899. https://doi.org/10.1109/BigData.2018.8622141

  3. Singha J, Das K (2013) Recognition of Indian sign language in live video. Int J Comput Appl 70(19):17–22. https://doi.org/10.5120/12174-7306

  4. Rivera-Acosta M, Ortega-Cisneros S, Rivera J, Sandoval-Ibarra F (2017) American sign language alphabet recognition using a neuromorphic sensor and an artificial neural network. Sensors 17(10). https://doi.org/10.3390/s17102176

  5. Patel R (2018) A review on image based indian sign language recognition. Int J Innov Res Comput Commun Eng, 9101–9107. https://doi.org/10.15680/IJIRCCE.2018

  6. Truong VNT, Yang CK, Tran QV (2016) A translator for american sign language to text and speech. 2016 IEEE 5th Glob. Conf Consum Electron. https://doi.org/10.1109/GCCE.2016.7800427

  7. Taskiran M, Killioglu M, Kahraman N (2018) A real-time system for recognition of american sign language by using deep learning. 41st Int Conf Telecommun Signal Process, 1–5. https://doi.org/10.1109/TSP.2018.8441304

  8. Ravikiran J, Mahesh K, Mahishi S, Dheeraj R, Sudheender S, Pujari NV (2010) Automatic recognition of sign language images. Lect Notes Electr Eng 52 LNEE:321–332. https://doi.org/10.1007/978-90-481-3517-2-25

  9. Rao GA, Syamala K, Kishore PVV, Sastry ASCS (2018) Deep convolutional neural networks for sign language recognition. Conference on Signal Processing and Communication Engineering Systems (SPACES), 194–197. https://doi.org/10.1109/SPACES.2018.8316344

  10. Wadhawan A, Kumar P (2020) Deep learning-based sign language recognition system for static signs. Neural Comput Appl: 0123456789. https://doi.org/10.1007/s00521-019-04691-y

  11. Tesfaye M (2010) Machine translation approach to translate amharic text to ethiopian sign language. St. Mary’s University College, pp 252–273

  12. Gimibi T (2014) Recognition of isolated signs in Ethiopian sign language Tefera Gimbi Recognition of isolated signs in Ethiopian sign language. Addis Ababa University, p 174

  13. Admasu YF, Raimond K (2010) Ethiopian sign language recognition using artificial neural network. Proc 2010 10th Int Conf Intell Syst Des Appl, 995–1000. https://doi.org/10.1109/ISDA.2010.5687057

  14. Tamiru NK (2018) Amharic Sign language recognition based on amharic alphabet signs. Addis Ababa University, p 201

  15. Zerubabel L (2008) Ethiopian finger spelling classification: a study to automate ethiopian sign language. Addis Ababa University, p 203

  16. Salau AO, Jain S (2019) Feature extraction: a survey of the types, techniques, and applications. 5th IEEE International Conference on Signal Processing and Communication (ICSC), Noida, India, 158–164. https://doi.org/10.1109/ICSC45622.2019.8938371

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, College of Computing and Informatics, Haramaya University, Dire Dawa, Ethiopia

    Bekalu Tadele Abeje

  2. Department of Electrical/Electronics and Computer Engineering, Afe Babalola University, Ado-Ekiti, Nigeria

    Ayodeji Olalekan Salau

  3. Saveetha School of Engineering, Saveetha Institute of Medical and Technical Sciences, Chennai, India

    Ayodeji Olalekan Salau

  4. Department of Computer Science, Institute of Technology, Bahir Dar University, Bahir Dar, Ethiopia

    Abreham Debasu Mengistu

  5. School of Electrical and Computer Engineering, Institute of Technology, Debre Markos University, Debre Markos, Ethiopia

    Nigus Kefyalew Tamiru

Authors
  1. Bekalu Tadele Abeje
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayodeji Olalekan Salau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abreham Debasu Mengistu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nigus Kefyalew Tamiru
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ayodeji Olalekan Salau.

Ethics declarations

Conflicts of interest/Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abeje, B.T., Salau, A.O., Mengistu, A.D. et al. Ethiopian sign language recognition using deep convolutional neural network. Multimed Tools Appl 81, 29027–29043 (2022). https://doi.org/10.1007/s11042-022-12768-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12768-5

Keywords

Search

Navigation