Abstract
We present a neural network based system for the visual recognition of human hand pointing gestures from stereo pairs of video camera images. The accuracy of the current system allows to estimate the pointing target to an accuracy of 2 cm in a workspace area of 50×50 cm. The system consists of several neural networks that perform the tasks of image segmentation, estimation of hand location, estimation of 3D-pointing direction and necessary coordinate transforms. Drawing heavily on the use of learning algorithms, the functions of all network modules were created from data examples only.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
S.J.Nowlan and J.C.Platt. “A convolutional neural network hand tracker”, in G.Tesauro, D.S.Touretzky and T.K.Leen (eds) Neural Information Processing Systems 7, MIT Press: Cambridge, MA, 1995.
T.J.Darell and A.P.Pentland, “Classifying hand gestures with a view-based distributed representation”, in J.D.Cowan, G.Tesauro, and J.Alspector (eds) Neural Information Processing Systems 6, pp. 945–953, Morgan Kaufman: San Mateo, CA, 1994.
J.Davis and M.Shah, “Recognizing hand gestures’, in J.-O.Eklundh (ed) Computer Vision-ECCV '94, Vol. 800 of Lecture Notes in Computer Science, pp. 331–340, Springer-Verlag: Berlin, Heidelberg, New York, 1994.
C. Maggioni, “A novel device for using the hand as a human-computer interface”, in Proc. HCI'93-Human Control Interface, Loughborough, UK, 1993.
A.Meyering and H.Ritter, “Learning 3D hand postures from perspective pixel images”, in I.Aleksander and J.Taylor (eds) Artificial Neural Networks 2, pp. 821–824, Elsevier Science Publishers: Amsterdam, 1992.
F.Kummert, E.Littmann, A.Meyering, S.Posch, H.Ritter and G.Sagerer, “Recognition of 3d-hand orientation from monocular color images by neural semantic networks”, Pattern Recognition and Image Analysis, Vol. 3, No. 3, pp. 311–316, 1993.
E.Littmann and H.Ritter, “Neural and statistical methods for adaptive color segmentation-a comparison”, in G.Sagerer, S.Posch, and F.Kummert (eds) Mustererkennung 1995, pp. 84–93, Springer-Verlag: Heidelberg, 1995.
A.Drees, F.Kummert, E.Littmann, S.Posch, H.Ritter and G.Sagerer, “A hybrid system to detect hand orientation in stereo images”, in E.S.Gelsema and L.N.Kanal (eds) Pattern Recognition in Practice IV: Multiple Paradigms, Comparative Studies and Hybrid Systems, pp. 551–562, Elsevier Science Publishers: Amsterdam, 1994.
H.Ritter, “Learning with the self-organizing map”, in T.Kohonen, K.Mäkisara, O.Simula and J.Kangas (eds) Artificial neural Networks 1, pp. 357–364. Elsevier Science Publishers: Amsterdam, 1991.
H.Ritter, T.Martinetz, and K.Schulten, Neural Computation and Self-Organizing Maps: An Introduction, Addison-Wesley: New York, 1992 (English and German).
T.Kohonen, Self Organization and Associative Memory, Springer Series in Information Sciences 8, Springer-Verlag: Berlin, Heidelberg, New York, 1982 (3rd edition 1989).
T.Kohonen, “The self-organizing map”, Proc. IEEE., Vol. 78, pp. 1464–1480, 1990.
T.Poggio and F.Girosi, “Regularization algorithms for learning that are equivalent to multilayer networks”, Science, Vol. 247, pp. 978–982, 1990.
J.D.Farmer and J.J.Sidorowich, “Predicting chaotic time series”, Physical Review Letters, Vol. 59, pp. 845–848, 1987.
K.Stokbro, D.K.Umberger and J.A.Hertz, “Exploiting neurons with localized receptive fields to learn chaos”, Complex Systems, Vol. 4, pp. 603–622, 1990.
C.G.Atkeson, “Memory-based approaches to approximating continuous functions”, in M.Casdagli and S.Eubank (eds) Nonlinear Modeling and Forecasting Vol. XII of SFI Studies in the Sciences of Complexity, pp. 503–521, Addison-Wesley: New York, 1992.
H.Ritter, “Parametrized self-organizing maps for vision learning tasks”, in P.Morasso (ed) ICANN '94, Springer-Verlag: Berlin, Heidelberg, New York, 1994.
J. Walter and H. Ritter, “Rapid learning with parametrized self-organizing maps”, Neural Computing, 1996 (in press).
E.Littmann, Strukturierung Neuronaler Netze zwischen Biologie und Anwendung, DISKI, Infix Verlag: St. Augustin, Germany, 1995.
A. Meyering and H. Ritter, “Learning 3D shape perception with local linear maps” in Proceedings of the International Joint Conference on Neural Networks, Vol. IV, pp. 432–436, Baltimore MD, June 7–11, 1992.
F.Kummert, E.Littmann, A.Meyering, S.Posch, H.Ritter and G.Sagerer, “A hybrid approach to signal interpretation using neural and semantic networks”, in S.J.Pöppl and H.Handels (eds) Mustererkennung 1993, pp. 245–252, Springer-Verlag: Berlin, Heidelberg, New York, 1993.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Litmann, E., Drees, A. & Ritter, H. Neural recognition of human pointing gestures in real images. Neural Process Lett 3, 61–71 (1996). https://doi.org/10.1007/BF00571679
Issue Date:
DOI: https://doi.org/10.1007/BF00571679