Abstract
Recently, language acquisition with aids of multi-modal information have drawn more and more attention. However, semantic grounding of verbs has been less concerned due to their complex semantic representation. This paper proposed a novel way to combine visual information into semantic representation of Chinese verb. While introducing original representation of two constituents, which are verb frame and argument from Frame Semantic, both of them are linked with visual information for verb semantic. And a visual information based categorization for arguments is mainly discussed. For achieving it, a collection of {video, its text description} pairs is first built. After preprocessing on both sides, the correspondence between arguments of verbs and related visual features is constructed basing on SOM groups. A video describing system has also been built to generate sentences for new videos. The evaluation of the describing system shows the effectiveness of our visual semantic representation on Chinese verbs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Palmer, M., Kingsbury, P., Gildea, D.: The Proposition Bank: An Annotated Corpus of Semantic Roles. Computational Linguistics 31(1), 71–106 (2005)
Sameer, S., Pradhan, E.H.H., Marcus, M.P., et al.: Ontonotes: a Unified Relational Semantic Representation. Int. J. Semantic Computing 1(4), 405–419 (2007)
Collin, F., Baker, C.J.F., Lowe, J.B.: The Berkeley FrameNet project. In: Proceedings of the 26 Annual Meeting of the ACL and 7th International Conference on Computational Linguistics, San Francisco, California (1998)
Sameer Pradhan, V.K.W.W., Jurafsky, D., Martin, J.H.: Using Semantic Representations in Question Answering. In: ICON 2002, Bombay, India (2002)
Karin Kipper Schuler, A.K.: VerbNet overview, extensions, mappings and applications. In: NAACL 2009, Boulder, Colorado (2009)
Ballard, C.Y.a.D.H.: On the Integration of Grounding Language and Learning Objects. American Association for Artificial Intelligence (2004)
Kubat, R.: Semantic Context Effects on Color Categorization. In: Proceedings of the 31st Annual Cognitive Science Society Meeting, Amsterdam (2009)
Zhang, C., Zhang, W., Liu, H., Wang, X.: Visual Information based Meaning Acquisition of Chinese. In: 10th Chinese National Conference on Computational Linguistics, Shan Dong Province, China, pp. 260–266 (2009)
Gentner: Why nouns are learned before verbs: Linguistic relativity versus natural partitioning. In: Language Development, NJ (1982)
Gleitman, L.: The structural sources of verb meanings. Language Acquisition (1990)
Jane Gillette, H.G., Gleitman, L., Lederer, A.: Human simulations of vocabulary learning. Cognition 73, 135–176 (1999)
Siskind, J.M.: Grounding the Lexical Semantics of Verbs in Visual Perception using Force Dynamics and Event Logic. Journal of Artificial Intelligence Research 15, 31–90 (2001)
Pangburn, et al.: EBLA: a perceptually grounded model of language acquisition. In: Proceedings of the HLT-NAACL, NJ, USA (2003)
Fleischman, M., Roy, D.: Unsupervised content-based indexing of sports video. In: International Multimedia Conference, Augsburg, Germany (2007)
Mukerjee, G.S.: Acquiring Linguistic Argument Structure from Multimodal Input using Attentive Focus. In: Development and Learning, ICDL 2008, Monterey, CA (2008)
Talmy, L.: Toward a Cognitive Semantics. MIT Press, Massachusetts (2000)
Fillmore, C.J.: Frame semantics, in Linguistics in the Morning Calm, e. In: The Linguistic Society of Korea, Seoul, pp. 111–137 (1982)
Collins, M.: A new statistical parser based on bigram lexical dependencies. In: Proceedings of the 34th Annual Meeting of the Association of Computational Linguistics, Santa Cruz, CA (1996)
Kohonen, T.: Automatic formation of topological maps of patterns in a self-organizing system. In: Proc. 2SCIA, Scand. Conf. on Image Analysis, Helsinki, Finland (1981)
Jain, J.M.a.A.K.: Artificial Neural Networks for Feature Extraction and Multivariate Data Projection. IEEE Transactions on Neural Networks 6(2), 296–317 (1995)
Jorma, T., Laaksonen, J.M.K., Erkki, O.: Class distributions on SOM surfaces for feature extraction and object retrieval. Neural Networks 17, 1121–1133 (2004)
Rousset, P., Guinot, C.: Distance between Kohonen Classes Visualization Tool to Use SOM in Data Set Analysis and Representation. In: Mira, J., Prieto, A.G. (eds.) IWANN 2001. LNCS, vol. 2085, p. 119. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, H., Wang, X., Zhong, Y. (2011). Visual Information Based Argument Categorization for Semantics of Chinese Verb. In: Park, J.J., Yang, L.T., Lee, C. (eds) Future Information Technology. Communications in Computer and Information Science, vol 185. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22309-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-22309-9_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22308-2
Online ISBN: 978-3-642-22309-9
eBook Packages: Computer ScienceComputer Science (R0)