An Intelligent Discovery and Error Correction Algorithm for Misunderstanding Gesture based on Probabilistic Statistics Model

doi:10.23940/ijpe.18.01.p10.89100

Abstract

Abstract:

Numerous experiments have shown that there are similar gestures in visual-based gesture recognition. In order to solve the problem, this paper proposes a new algorithm based on Convolutional Neural Network. According to the model test results, the confusion matrix is established. According to the correspondence between each gesture and the predicted result, probability matrix of misjudgement is established. Based on the probability matrix of misjudgement, we correct the gestures that have been incorrectly identified by the Convolutional Neural Network Model. After this algorithm, the recognition rate of similar gestures is increased from 5% to 12%. The innovation of this paper lies in the secondary error correction of the wrong gesture of Convolutional Neural Network structure.

Submitted on October 20, 2017; Revised on November 23, 2017; Accepted on December 7, 2017
References: 54

Kaiyun Sun, Zhiquan Feng, Changsheng Ai, Yingjun Li, Jun Wei, Xiaohui Yang, Xiaopei Guo, Hong Liu, Yanbin Han, and Yongguo Zhao. An Intelligent Discovery and Error Correction Algorithm for Misunderstanding Gesture based on Probabilistic Statistics Model [J]. Int J Performability Eng, 2018, 14(1): 89-100.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References 0

	1. M. K. Bhuyan, D. A. Kumar and K. F. Macdorman, “A Novel Set of Features for Continuous Hand Gesture Recognition,” Journal on Multimodal User Interfaces, vol. 8, no. 4, pp. 333-343, 2014
	2. P. Barros, S. Magg and C. Weber, “A Multichannel Convolutional Neural Network for Hand Posture Recognition,” Artificial Neural Networks and Machine Learning – ICANN 2014, pp. 403-410 2014
	3. C. C. Chang, J. J. Chen, and W. K. Tai, “New Approach for Static Gesture Recognition,” Journal of information science and engineering, vol. 22, no. 5, pp. 1047-1057, 2006
	4. J. Cao, X. L. Zhao, J.H. Wang, “Animated Gesture Recognition Method Based on RGB-D,” Journal of Computer Applications, no. 6, pp. 1-7, 2018
	5. C. Dong, C. C. Loy, and K. He, “Learning a Deep Convolutional Network for Image Super-resolution.” European Conference on Computer Vision, Springer, Cham, pp. 184-199, 2014
	6. C. S. Fahn, and K. Y. Chu, “Hidden-Markov-Model-Based Hand Gesture Recognition Techniques Used for a Human-Robot Interaction System,” Human-Computer Interaction, Interaction Techniques and Environments, Springer Berlin Heidelberg, pp. 1043-1058, 2010
	7. S. Fong, Z. Yan, I. Fister “A Biometric Authentication Model using Hand Gesture Images,” BioMedical Engineering OnLine, vol. 12, no. 1, pp. 1-18 2013
	8. W. T. Freeman, and M. Roth, “Orientation Histograms for Hand Gesture Recognition,” International workshop on automatic face and gesture recognition, vol. 12, pp. 296-301, 1995
	9. R. Girshick, “Fast R-cnn,” Proceedings of the IEEE international conference on computer vision, pp. 1440-1448, 2015
	10. A. R. Hafiz, M. F. Amin, and K. Murase, “Real-time Hand Gesture Recognition using Complex-valued Neural Network,” International conference on neural information processing, pp. 541-549, 2011
	11. H. Haitham, S. Abdul-Kareem, “Static Hand Gesture Recognition using Neural Networks,” Artificial Intelligence Review, pp. 1-35, 2012
	12. H, S. Hasan, S. B. A. Kareem, “Gesture Feature Extraction for Static Gesture Recognition,” Arabian Journal for Science & Engineering (Springer Science & Business Media BV), vol. 38, no. 12, 2013
	13. S. Ji, W. Xu, and M.Yang, “3D Convolutional Neural Networks for Human Action Recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 1, pp. 221-231, 2013
	14. Y. Jia, E. Shelhamer, J. Donahue “Caffe: Convolutional Architecture for Fast Feature Embedding” Proceedings of the 22nd ACM international conference on Multimedia, PP. 675-678, 2014
	15. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet Classification with Deep Convolutional Neural Networks” Advances in neural information processing systems, pp. 1097-1105,2012
	16. S. Lawrence, C. L. Giles, and A. C. Tsoi, “Face Recognition: A Convolutional Neural-Network Approach,” IEEE transactions on neural networks, vol. 8, no. 1, pp. 98-113, 1997
	17. S. Li, and Z. Q. Liu, and A. B.Chan, “Hetero Geneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 482-489, 2014
	18. Y. T. Li, J.P. Wachs, “HEGM: A Hierarchical Elastic Graph Matching for Hand Gesture Recognition,” Pattern Recognition, vol. 47, no. 1, pp. 80-88, 2014
	19. G. Marin, F. Dominio, and P. Zanuttigh, “Hand Gesture Recognition with Leap Motion and Kinect Devices,” Image Processing (ICIP), 2014 IEEE International Conference on, pp. 1565-1569, 2014
	20. P. Molchanov, S. Gupta, and K. Kim, “Hand Gesture Recognition with 3D Convolutional Neural Networks” Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 1-7, 2003
	21. S. Mitra, T. and Acharya Wu, “Gesture Recognition: A Survey,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 37, no. 3, pp. 311-324, 2007
	22. Z. Mo and U. Neumann, “Lexical Gesture Interface,” IEEE International Conference on. IEEE, pp. 7-7, 2006
	23. J. Nagi, F. Ducatelle, and G. A. Di Caro, “Max-pooling Convolutional Neural Networks for Vision-based Hand Gesture Recognition,” Signal and Image Processing Applications (ICSIPA), 2011 IEEE International Conference on, pp. 342-347, 2011
	24. T. N. Nguyen, M. M. Huynh, and J. Meunier, “Static Hand Gesture Recognition using Artificial Neural Network,” Journal of Image and Graphics, vol. 1, no. 1, pp. 34-38, 2013
	25. M. Oquab, L. Bottou and I. Laptev, “Learning and Transferring Mid-level Image Representations using Convolutional Neural Networks,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1717-1724 ,2014
	26. L. Pigou, S. Dieleman, and P. J. Kindermans, “Sign Language Recognition using Convolutional Neural Networks,” Workshop at the European Conference on Computer Vision, Springer, Cham, pp. 572-578, 2014
	27. O. Patsadu, C. Nukoolkit, and B. Watanapa, “Human Gesture Recognition using Kinect Camera,” Computer Science and Software Engineering (JCSSE), 2012 International Joint Conference on, pp. 28-32, 2012
	28. Q. Pu, S. Gupta, and S. Gollakota, “Whole-home Gesture Recognition using Wireless Signals,” Proceedings of the 19th annual international conference on Mobile computing & networking, pp. 27-38, 2013
	29. S. P. Priyal, and P. K. Bora, “A Robust Static Hand Gesture Recognition System using Geometry Based Normalizations and Krawtchouk Moments,” Pattern Recognition, vol. 46, no. 8, pp. 2202-2219, 2013
	30. T. Pfister, J. Charles, and A. Zisserman, “Flowing Convnets for Human Pose Estimation in Videos,” Proceedings of the IEEE International Conference on Computer Vision, pp. 1913-1921, 2015
	31. V. Pitsikalis, A. Katsamanis, and S. Theodorakis, “Multimodal Gesture Recognition Via Multiple Hypotheses Rescoring,” Gesture Recognition. Springer, Cham, pp. 467-496, 2017
	32. S. Qin, X. Zhu, and H. Yu, “Real-time Markerless Hand Gesture Recognition with Depth Camera,” Advances in Multimedia Information Processing – PCM 2012, pp. 186-197, 2012
	33. S. S. Rautaray,and A. Agrawal, “Vision Based Hand Gesture Recognition for Human Computer Interaction: A Survey,” Artificial Intelligence Review, vol. 43, no. 1, pp. 1-54, 2015
	34. S. Ren, K. He, and R. Girshick, “Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks,” Advances in neural information processing systems, pp. 91-99, 2015
	35. T. Roska, L. O. Chua, and G. Wen, “The CNN Universal Machine: An Analogic Array Computer,” J IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, vol. 40, no. 3, pp. 163-173, 1993
	36. Z. Ren, J. Yuan, and J. Meng, “Robust Part-based Hand Gesture Recognition using Kinect Sensor,” IEEE transactions on multimedia, vol. 15, no. 15, pp. 1110-1120, 2013
	37. Z. Ren, J. Yuan, and Z. Zhang, “Robust Hand Gesture Recognition Based on Finger-earth Mover's Distance with a Commodity Depth Camera,” Proceedings of the 19th ACM international conference on Multimedia, pp. 1093-1096, 2011
	38. A. Sharif Razavian, H. Azizpour, and H. Sullivan, “CNN Features Off-the-shelf: An Astounding Baseline for Recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 806-813, 2014
	39. C. Szegedy, W. Liu, and Y. Jia, “Going Deeper with Convolutions,” Proceedings of the IEEE conference on computer vision and pattern recognition, pp.1-9, 2015
	40. E. Stergiopoulou, and N. Papamarkos, “Hand Gesture Recognition using a Neural Network Shape Fitting Technique,” Engineering Applications of Artificial Intelligence, vol. 22, no. 8, pp. 1141-1158, 2009
	41. J. Suarez, and R. R. Murphy, “Hand Gesture Recognition with Depth Images: A Review,” Ro-man, 2012 IEEE, pp. 411-417, 2012
	42. k. Simonyan, and A. Zisserman, “Very Deep Convolutional Networks for Large-scale Image Recognition,” arXiv preprint arXiv, pp.1049, 1556, 2014
	43. P. Y. Simard, D. Steinkraus, and J. C. Platt, “Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis,” ICDAR, vol. 3, pp. 958-962, 2003
	44. M. Turk, “Gesture Recognition,” Computer Vision: A Reference Guide, pp. 346-349, 2014
	45. M. F. Tolba, A. Samir, and M. Aboul-Ela, “Arabic Sign Language Continuous Sentences Recognition using PCNN and Graph Matching,” Neural Computing and Applications, vol. 23, no. 3, pp. 999-1010, 2013
	46. L. Wang, H. Liu, B. Wang, and P. Li, “Gesture Recognition Method Combined with Skin Color Model and Convolution Neural Network,” Computer Engineering and Applications, no. 6, pp. 209-214, 2017
	47. L. Xiao, R. Li, F.Z. Zeng, “Animated Gesture Recognition method Based on Self-learning Sparse Representation,” Journals of Communications, vol. 34, no. 6, pp. 128-135, 2013
	48. L. Xu, J. S. J. Ren, and C. Liu, “Deep Convolutional Neural Network for Image Deconvolution,” Advances in Neural Information Processing Systems, pp. 1790-1798, 2014
	49. R. Xie, X. Sun, X. Xia, “Similarity Matching-Based Extensible Hand Gesture Recognition,” IEEE Sensors Journal, vol. 15, no. 6, pp. 3475-3483, 2015
	51. B. Yang, X. N. Song, Z. Q. Feng, and S. Yan, “A Gesture Recognition Algorithm Based on Spatial Distribution Feature in Complex Background,” Journal of Computer Aided design and graphic Science, no. 10, pp. 1841-1848, 2010
	52. H. S. Yeo, B. G. Lee, and H. Lim, “Hand Tracking and Gesture Recognition System for Human-computer Interaction using Low-cost Hardware,” Multimedia Tools and Applications, vol. 74, no. 8, pp. 2687-2715, 2015
	53. H. Zheng, X.K. Shen, “Animated Gesture Recognition Method Based on Self-learning Sparse Representation,” Beijing University of Aeronautics and Astronautics, vol. 38, no. 2, pp. 273-279, 2012
	54. L. G. Zhang, J. Q. Wu, and G. Wen, “Hand Gesture Recognition Based on Hausdorff Distance,” Journal of Image and Graphics: A, vol. 7, no. 11, pp. 1144-1150, 2002