Skip to main content
SpringerLink
Log in

Integrated features and GMM Based Hand Detector Applied to Character Recognition System under Practical Conditions

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Detection of bare-hand under non-ideal conditions is a challenging task. Most of the existing hand detection systems are developed under limited environmental constraints. In this study, a robust two-level bare-hand detector is integrated with a 58 keyboard characters recognition model. At first, the Gaussian mixture model (GMM) based foreground detector is used to segment the region of interest (ROI), which is further classified using Color-texture and texture based models to detect the actual fist. The detected hand is tracked using modified Kanade–Lucas–Tomasi (KLT) tracker to generate the required trajectory points of the character. The feature space for character recognition consists of existing features and three new features, namely, Local Geometrical Area Ratio (LGAR), Area of two halves (ATH), Curve-Area feature (CAF) that are extracted from the trajectory points. Feature space is optimized using statistical analysis algorithms. Multi-factor analysis of individual character subsets such as alphabets, numbers, ASCII characters, etc., are carried out using multiple conventional classifiers along with Support vector machine (SVM), extreme learning machine (ELM), artificial neural network (ANN), and proposed Neuro-fuzzy classifiers. The proposed GMM based motion detection method achieves an accuracy of 100% during the segmentation of ROI, followed by an increase of 46.77% in the accuracy of two-level hand detection under non-ideal conditions. Maximum accuracy of 58 character system using proposed features and ANN classifier is observed to be 92.56%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Barros P, Maciel-Junior NT, Fernandes BJ, BezerraBL FSM (2017) A dynamic gesture recognition and prediction system using the convexity approach. Comput Vis Image Underst 155:139–149

    Article  Google Scholar 

  2. Bengio S, Bourlard H (2005) Machine learning for multimodal interaction. Springer, Berlin, Heidelberg

    Book  Google Scholar 

  3. Bhuyan MK, Kumar DA, MacDorman KF, Iwahori Y (2014) A novel set of features for continuous hand gesture recognition. Journal on Multimodal User Interfaces 8(4):333–343

    Article  Google Scholar 

  4. Bianconi F, Harvey RW, Southam P, Fernández A (2011) Theoretical and experimental comparison of different approaches for color texture classification. J Electron Imaging 20(4):043006

    Article  Google Scholar 

  5. Cetisli B (2010) Development of an adaptive neuro-fuzzy classifier using linguistic hedges: Part 1. Expert Syst Appl 37(8):6093–6101

    Article  Google Scholar 

  6. Cetisli B (2010) The effect of linguistic hedges on feature selection: Part 2. Expert Syst Appl 37(8):6102–6108

    Article  Google Scholar 

  7. Cetişli B, Barkana A (2010) Speeding up the scaled conjugate gradient algorithm and its application in neuro-fuzzy classifier training. Soft Comput 14(4):365

    Article  MATH  Google Scholar 

  8. Chen Q, Georganas ND, and Petriu EM (2007) Real-time vision-based hand gesture recognition using haar-like features. In Instrumentation and Measurement Technology Conference Proceedings, 2007. IMTC 2007, IEEE. pp. 1–6

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition. IEEE Computer Society Conference 1:886–893

    Google Scholar 

  10. Ding H, Feng PM, ChenW LH (2014) Identification of bacteriophage virion proteins by the ANOVA feature selection and analysis. Mol BioSyst 10(8):2229–2235

    Article  Google Scholar 

  11. Douglas DH, and Peucker TK (2011) Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or its Caricature. Classics in Cartography: Reflections on Influential Articles from Cartographica, Wiley Online Library, pp. 15–28

  12. Dürre J, Paradzik D, Blume H (2018) A HOG-based Real-time and Multi-scale Pedestrian Detector Demonstration System on FPGA. In Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pp. 163–172

  13. Elmezain M, Al-Hamadi A, Krell G, El-Etriby S, and Michaelis B (2007) Gesture recognition for alphabets from hand motion trajectory using hidden markov models. In 2007 IEEE international symposium on signal processing and information technology IEEE, pp. 1192–1197

  14. Friedman J, Hastie T, Tibshirani R (2000) Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann Stat 28(2):337–407

    Article  MathSciNet  MATH  Google Scholar 

  15. Haria A, Subramanian A, Asokkumar N, Poddar S, Nayak JS (2017) Hand gesture recognition for human computer interaction. Proc Comput Sci 115:367–374

    Article  Google Scholar 

  16. Jian M, Lam KM (2014) Face-image retrieval based on singular values and potential-field representation. Signal Process 100:9–15

    Article  Google Scholar 

  17. Jian M, Lam KM (2015) Simultaneous hallucination and recognition of low-resolution faces based on singular value decomposition. IEEE Trans Circuits Syst Video Technol 25(11):1761–1772

    Article  Google Scholar 

  18. Jian M, Lam KM, Dong J, Shen L (2014) Visual-patch-attention-aware saliency detection. IEEE trans cybern 45(8):1575–1586

    Article  Google Scholar 

  19. Jian M, Qi Q, Dong J, Sun X, Sun Y, Lam KM (2018) Saliency detection using quaternionic distance based weber local descriptor and level priors. Multimed Tools Appl 77(11):14343–14360

    Article  Google Scholar 

  20. Jian M, Zhang W, Yu H, Cui C, Nie X, Zhang H, Yin Y (2018) Saliency detection based on directional patches extraction and principal local color contrast. J Vis Commun Image Represent 57:1–11

    Article  Google Scholar 

  21. KaewTraKulPong P, Bowden R (2002) An improved adaptive background mixture model for real-time tracking with shadow detection. In Video-based surveillance systems, Springer, pp. 135–144

  22. Kakumanu P, Makrogiannis S, Bourbakis N (2007) A survey of skin-color modeling and detection methods. Pattern Recogn, Elsevier 40(3):1106–1122

    Article  MATH  Google Scholar 

  23. Kao CY, Fahn CS (2011) A human-machine interaction technique: hand gesture recognition based on hidden Markov models with trajectory of hand motion. Process Eng 15:3739–3743

    Google Scholar 

  24. Karam M (2006) A framework for research and design of gesture-based human computer interactions. PhD Thesis, University of Southampton

  25. Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20:226–239

    Article  Google Scholar 

  26. Kobayashi T, Otsu N (2008) Image feature extraction using gradient local auto-correlations. In: European conference on computer vision. Springer, Berlin, Heidelberg, pp 346–358

    Google Scholar 

  27. Kodituwakku SR, Selvarajah S (2004) Comparison of color features for image retrieval. Indian J Comput Sci Eng 1(3):207–211

    Google Scholar 

  28. Kuncheva LI (2002) A theoretical study on six classifier fusion strategies. IEEE Trans Pattern Anal Mach Intell 24(2):281–286

    Article  Google Scholar 

  29. Lepistö L, Kunttu I, Visa A (2005) Color-based classification of natural rock images using classifier combinations. Lect Notes Comput Sci 3540:901–909

    Article  Google Scholar 

  30. Li C, Xie C, Zhang B, Chen C, Han J (2018) Deep Fisher discriminant learning for mobile hand gesture recognition. Pattern Recogn 77:276–288

    Article  Google Scholar 

  31. Lin J, Ding Y (2013) A temporal hand gesture recognition system based on hog and motion trajectory. Optik 124(24):6795–6798

    Article  Google Scholar 

  32. Liu Y, Wang X, Yan K (2018) Hand gesture recognition based on concentric circular scan lines and weighted K-nearest neighbor algorithm. Multimed Tools Appl 77(1):209–223

    Article  Google Scholar 

  33. Lv Z, Halawani A, Feng S, Ur Réhman S, Li H (2015) Touch-less interactive augmented reality game on vision-based wearable device. Pers Ubiquit Comput 19(3–4):551–567

    Article  Google Scholar 

  34. McCue R (2009) A Comparison of the Accuracy of Support Vector Machine and Naıve Bayes Algorithms. In Spam Classification

  35. Misra S, Laskar RH (2017) Multi-factor analysis of texture and color-texture features for robust hand detection in non-ideal conditions. In Region 10 Conference, TENCON 2017–2017 IEEE, pp. 1165–1170

  36. Misra A, Abe T, Deguchi K (2011) Hand Gesture Recognition Using Histogram of Oriented Gradients and Partial Least Squares Regression. In MVA, pp. 479–482

  37. Misra S, Singha J, Laskar RH (2017) Vision-based hand gesture recognition of alphabets, numbers, arithmetic operators and ASCII characters in order to develop a virtual text-entry interface system. Neural Computing and Applications, Springer, pp. 1–19

  38. Pietik¨ainen M, Nieminen S, Marszalec E, and Ojala T (1996) Accurate color discrimination with classification based on features distributions, In Proceedings of the 13th International Conference on Pattern Recognition, 3, :833–838

  39. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54

    Article  Google Scholar 

  40. Rekha J, Bhattacharya J, Majumder S (2011) Hand gesture recognition for sign language: A new hybrid approach In Proc. Int. Conf. Image Processing, Computer Vision, and Pattern Recognition, pp. 80–86

  41. Schapire RE (1990) The strength of weak learnability. Mach learn, Springer 5(2):197–227

    Google Scholar 

  42. Schapire RE, Singer Y (1998) Improved boosting algorithms using confidence-rated predictions. In Proceedings of the eleventh annual conference on Computational learning theory, ACM, pp. 80–91

  43. Singh C, Walia E, and Kaur KP (2017) Color Texture Description with Novel Local Binary Patterns for Effective Image Retrieval. Pattern Recognition

  44. Singha J, Laskar RH (2016) Recognition of global hand gestures using self co-articulation information and classifier fusion. Journal on Multimodal User Interfaces 10(1):77–93

    Article  Google Scholar 

  45. Singha J, Roy A, and Laskar RH (2015) Dynamic hand gesture recognition using vision-based approach for human–computer interaction. Neural Computing and Applications, Springer. pp. 1–3

  46. Stančić I, Musić J, Grujić T (2017) Gesture recognition system for real-time mobile robot control based on inertial sensors and motion strings. Eng Appl Artif Intell 66:33–48

    Article  Google Scholar 

  47. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on, vol 2, pp 246–252

    Google Scholar 

  48. Sun CT, Jang JS (1993) A neuro-fuzzy classifier and its applications. In Fuzzy Systems, Second IEEE International Conference, pp. 94–98

  49. Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32

    Article  Google Scholar 

  50. Tripathy DP, Reddy KGR (2017) Novel Methods for Separation of Gangue from Limestone and Coal using Multispectral and Joint Color-Texture Features. Journal of the Institution of Engineers (India): Series D 98(1):109–117

    Article  Google Scholar 

  51. Vezhnevets A, Vezhnevets V (2005) Modest AdaBoost-teaching AdaBoost to generalize better. In Graphicon. 12(5):987–997

  52. Wood JM (2007) Understanding and Computing Cohen’s Kappa: A Tutorial. WebPsychEmpiricist. Web Journal at http://wpe. info/. Accessed October 3, 2007

  53. Woods RE, Gonzalez RC (1981) Real-time digital image enhancement. Proc IEEE 69(5):643–654

    Article  Google Scholar 

  54. Xu Y, Gu J, Tao Z, Wu D (2009) Bare hand gesture recognition with a single color camera. In Image and Signal Processing, 2009. CISP'09. 2nd International Congress on IEEE, pp. 1–4

  55. Xu D, Wu X, Chen YL, Xu Y (2014) Online dynamic gesture recognition for human robot interaction. J Intell Rob System 77(3–4):583–596

    Google Scholar 

  56. Yang C, Han DK, Ko H (2017) Continuous hand gesture recognition based on trajectory shape information. Pattern Recogn Lett 99:39–47

    Article  Google Scholar 

  57. Yoon HS, Soh J, Bae YJ, Yang HS (2001) Hand gesture recognition using combined features of location, angle and velocity. Pattern Recogn 34(7):1491–1501

    Article  MATH  Google Scholar 

  58. Zaki MM, Shaheen SI (2011) Sign language recognition using a combination of new vision based features. Pattern Recogn Lett 32(4):572–577

    Article  Google Scholar 

  59. Zeng W, Wang C, and Wang Q (2018) Hand gesture recognition using Leap Motion via deterministic learning. Multimedia Tools and Applications, 1–22

Download references

Acknowledgments

Authors are thankful to Science & Engineering Research Board (SERB) under Department of Science & Technology (DST), Government of India for the financial support under IMPRINT-II project with Diary No. SERB/F/I0220/2018-2019. Authors are thankful to MEITY, Government of India for the financial support under the Visvesvaraya Ph.D. scheme with Grant no. PhD-MLA/4(74)/2015-16. The authors are also thankful to NIT Silchar and Aditya Engineering College, Surampelam, India for proving the authors with equipments for carrying out the research work.

Author information

Authors and Affiliations

  1. Department of ECE, Aditya Engineering College (A), Surampelam, Andhra Pradesh, 533437, India

    Songhita Misra

  2. Department of ECE, NIT Silchar, Silchar, Assam, 788010, India

    R. H. Laskar

Authors
  1. Songhita Misra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. H. Laskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Songhita Misra.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Misra, S., Laskar, R.H. Integrated features and GMM Based Hand Detector Applied to Character Recognition System under Practical Conditions. Multimed Tools Appl 78, 34927–34961 (2019). https://doi.org/10.1007/s11042-019-08105-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08105-y

Keywords

Search

Navigation