Abstract
The paper presents a project aiming at collecting, annotating and exploiting a dialogue corpus from a multimodal perspective. The goal of the project is the description of the different parameters involved in a natural interaction process. Describing such complex mechanism requires corpora annotated in different domains. This paper first presents the corpus and the scheme used in order to annotate the different domains that have to be taken into consideration, namely phonetics, morphology, syntax, prosody, discourse and gestures. Several examples illustrating the interest of such a resource are then proposed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allwood, J., Cerrato, L., Dybkjaer, L., et al.: The MUMIN Multimodal Coding Scheme, NorFA yearbook 2005 (2005), http://www.ling.gu.se/~jens/publications/B%20files/B70.pdf
Bertrand, R., Blache, P., Espesser, R., et al.: Le CID - Corpus of Interactional Data - Annotation et Exploitation Multimodale de Parole Conversationnelle. In revue Traitement Automatique des Langues 49(3) (2008)
Bertrand, R., Ferré, G., Blache, P., Espesser, R., Rauzy, S.: Backchannels revisited from a multimodal perspective. In: Proceedings of Auditory-visual Speech Processing (2007)
Blache, P., Rauzy, S.: Influence de la qualité de l’étiquetage sur le chunking: une corrélation dépendant de la taille des chunks. In: Proceedings of TALN 2008 (2008)
Blanche-Benveniste, C., Jeanjean, C.: Le français parlé, Transcription et édition, Didier (1987)
Brun, A., Cerisara, C., Fohr, D., Illina, I., Langlois, D., Mella, O., Smaïli, K.: Ants: le système de transcription automatique du Loria, in actes des XXVe JEP (2004)
Carletta, J., Isard, A.: The MATE Annotation Workbench: User Requirements. In: Proceedings of the ACL Workshop: Towards Standards and Tools for Discourse Tagging (1999)
Carletta, J., Evert, S., Heid, U., Kilgour, J., Robertson, J., Voormann, H.: The NITE XML Toolkit: flexible annotation for multi-modal language data. Behavior Research Methods, Instruments, and Computers 35(3) (2003)
Carletta, J.: Announcing the AMI Meeting Corpus. The ELRA Newsletter 11(1) (2006)
Carletta, J., Dingare, S., Nissim, M., Nikitina, T.: Using the NITE XML Toolkit on the Switchboard Corpus to study syntactic choice: a case study. In: Proceedings of LREC 2004 (2004)
Di Cristo, A., Di Cristo, P.: Syntaix, une approche métrique-autosegmentale de la prosodie. TAL 42(1), 69–114 (2001)
Di Cristo, A., Auran, C., Bertrand, R., et al.: Outils prosodiques et analyse du discours. In: Simon, A.C., Auchlin, A., Grobet, A. (eds.) Cahiers de Linguistique de Louvain 28, Peeters, pp. 27–84 (2004)
Dipper, S.: XML-based stand-off representation and exploitation of multi-level linguistic annotation. In: Proceedings of Berliner XML Tage, Berlin (September 2005)
Dipper, S., Götze, M., Skopeteas, S.: Information Structure in Cross-Linguistic Corpora: Annotation Guidelines for Phonology, Morphology, Syntax, Semantics, and Information Structure. Interdisciplinary Studies on In formation Structure, Working Papers of the SFB 632. University of Potsdam, vol. 7 (2007)
Ferré, G., Bertrand, R., Blache, P., Espesser, R., Rauzy, S.: Gestural Reinforcement of Degree Adverbs and Adjectives in French and English. In: Proceedings. of AFLICO (2009)
Ferré, G., Bertrand, R., Blache, P., Espesser, R., Rauzy, S.: Intensive Gestures in French and their Multimodal Correlates. In: Proceedings of Interspeech 2007 (2007)
Fraser, B.: What are discourse markers? Journal of Pragmatics 31 (1999)
Fox Tree, J.E.: Listening in on Monologues and Dialogues. Discourse Processes 27(1) (1999)
Hirst, D., Di Cristo, A., Espesser, R.: Levels of description and levels of representation in the analysis of intonation. In: Prosody: Theory and Experiment. Kluwer, Dordrecht (2000)
Hirst, D., Auran, C.: Analysis by synthesis of speech prosody: the ProZed environment. In: Proceedings of Interspeech/Eurospeech (2005)
Jun, S.-A., Fougeron, C.: Realizations of accentual phrase in French intonation. Probus 14 (2002)
Kendon, A.: Gesture: Visible Action As Utterance. Cambridge University Press, Cambridge (2004)
Kipp, M.: Gesture Generation By Imitation. From Human Behavior To Computer Character Animation, Florida, Boca Raton (2004), http://www.dfki.de/~Kipp/Dissertation.html
Krenn, B., Pirker, H.: Defining The Gesticon: Language And Gesture Coordination For Interacting Embodied Agents. In: Aisb 2004 Symposium On Language, Speech And Gesture For Expressive Characters (2004)
Kruijff-Korbayova, I., Gerstenberger, C., Rieser, V., Schehl, J.: The SAMMIE multimodal dialogue corpus meets the NITE XML toolkit. In: Proceedings of LREC 2006 (2006)
Loehr, D.P.: Gesture and Intonation. Doctoral Dissertation, Georgetown University (2004)
McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2005)
Norris, S.: Analyzing Multimodal Interaction. A Methodological Framework. Routledge, New York (2004)
Overstreet, M.: Whales, candlelight, and stuff like that: General extenders in English discourse. Oxford University Press, Oxford (1999)
Paroubek, P., Robba, I., Vilnat, A., Ayache, C.: Data Annotations and Measures in EASY the Evaluation Campaign for Parsers in French. In: Proceedings of LREC 2006 (2006)
Pineda, L.A., Massé, A., Meza, I., Salas, M., Schwarz, E., Uraga, E., Villaseñor, L.: The DIME Project. In: Coello Coello, C.A., de Albornoz, Á., Sucar, L.E., Battistutti, O.C. (eds.) MICAI 2002. LNCS (LNAI), vol. 2313, p. 166. Springer, Heidelberg (2002)
Rodriguez, K., Dipper, S., Götze, M., Poesio, M., Riccardi, G., Raymond, C., Rabiega-Wisniewska, J.: Standoff Coordination for Multi-Tool Annotation in a Dialogue Corpus. In: Proceedings of Linguistic Annotation Workshop (2007)
Schiffrin, D.: Discourse Markers. Cambridge University Press, Cambridge (1987)
Selting, M.: The construction of ’units’ in conversational talk, Language in Society 29 (2000)
Tusnelda, Tübingen collection of reusable, empirical, linguistic data structures (2005), http://www.sfb441.uni-tuebingen.de/tusnelda-engl.html
Vanrullen, T., Blache, P., Balfourier, J.-M.: Constraint-Based Parsing as an Efficient Solution: Results from the Parsing Evaluation Campaign EASy. In: Proceedings of LREC 2006 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Blache, P., Bertrand, R., Ferré, G. (2009). Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project. In: Kipp, M., Martin, JC., Paggio, P., Heylen, D. (eds) Multimodal Corpora. MMCorp 2008. Lecture Notes in Computer Science(), vol 5509. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04793-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-04793-0_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04792-3
Online ISBN: 978-3-642-04793-0
eBook Packages: Computer ScienceComputer Science (R0)