Crystal Chao
ABSTRACT
In human cooperation, the concurrent usage of multiple social modalities such as speech, gesture, and gaze results in robust and efficient communicative acts. Such multimodality in combination with reciprocal intentions supports fluent turn-taking. I hypothesize that human-robot turn-taking can be made more fluent through appropriate timing of multimodal actions. Managing timing includes understanding the impact that timing can have on interactions as well as having a control system that supports the manipulation of such timing. To this end, I propose to develop a computational turn-taking model of the timing and information flow of reciprocal interactions. I also propose to develop an architecture based on the timed Petri net (TPN) for the generation of coordinated multimodal behavior, inside of which the turn-taking model will regulate turn timing and action initiation and interruption in order to seize and yield control. Through user studies in multiple domains, I intend to demonstrate the system's generality and evaluate the system on balance of control, fluency, and task effectiveness.
- B. Blumberg, M. Downie, Y. Ivanov, M. Berlin, M. Johnson, and B. Tomlinson. Integrated learning for interactive synthetic characters. In Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), pages 417--426, 2002. Google Scholar
Digital Library
- D. Bohus and E. Horvitz. Facilitating multiparty dialog with gaze, gesture, and speech. In Proceedings of the 12th International Conference on Multimodal Interfaces (ICMI), 2010. Google ScholarDigital Library
- M. Cakmak, C. Chao, and A. L. Thomaz. Designing interactions for robot active learners. IEEE Transactions on Autonomous Mental Development, 2(2):108--118, June 2010. Google ScholarDigital Library
- J. Cassell, T. Bickmore, L. Campbell, K. Chang, H. Vilhjálmsson, and H. Yan. Requirements for an architecture for embodied conversational characters. In Computer Animation and Simulation, pages 109--120. Springer Verlag, 1999.Google Scholar
- C. Chao, J. H. Lee, M. Begum, and A. Thomaz. Simon plays Simon says: The timing of turn-taking in an imitation game. In IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2011.Google ScholarCross Ref
- C. Chao and A. Thomaz. Timing in multimodal reciprocal interactions: Control and analysis using timed Petri nets. Journal of Human-Robot Interaction, 1(1):46--67, 2012.Google Scholar
- H. H. Clark and M. A. Krych. Speaking while monitoring addressees for understanding. Journal of Memory and Language, 50(1):62--81, 2004.Google ScholarCross Ref
- S. Duncan. On the structure of speaker-auditor interaction during speaking turns. Language in Society, 3(2):161--180, 1974.Google ScholarCross Ref
- G. Hoffman and C. Breazeal. Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team. In Proceedings of the 3rd ACM/IEEE Conference on Human-Robot Interaction (HRI), 2007. Google ScholarDigital Library
- A. G. Holroyd. Generating engagement behaviors in human-robot interaction. Master's thesis, Worcester Polytechnic Institute, 2011.Google Scholar
- H. Kose-Bagci, K. Dautenhan, and C. L. Nehaniv. Emergent dynamics of turn-taking interaction in drumming games with a humanoid robot. In Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication, pages 346--353, 2008.Google ScholarCross Ref
- B. Mutlu, T. Shiwa, T. K. H. Ishiguro, and N. Hagita. Footing in human-robot conversations: How robots might shape participant roles using gaze cues. In Proceedings of the 4th ACM/IEEE Conference on Human-Robot Interaction (HRI), 2009. Google ScholarDigital Library
- B. Orestrom. Turn-taking in English conversation. CWK Gleerup, 1983.Google Scholar
- A. Raux and M. Eskenazi. A finite-state turn-taking model for spoken dialog systems. In Proceedings of the Human Language Technologies (HLT), 2009. Google ScholarDigital Library
- H. Sacks, E. Schegloff, and G. Jefferson. A simplest systematics for the organization of turn-taking for conversation. Language, 50:696--735, 1974.Google ScholarCross Ref
- E. Schegloff. Overlapping talk and the organization of turn-taking for conversation. Language in Society, 29(1):1--63, 2000.Google ScholarCross Ref
- E. Tronick, H. Als, and L. Adamson. Structure of early face-to-face communicative interactions. In M. Bullowa, editor, Before Speech: The Beginning of Interpersonal Communication, pages 349--374. Cambridge University Press, Cambridge, 1979.Google Scholar
Index Terms
- Timing multimodal turn-taking for human-robot cooperation
Recommendations
Smooth Turn-taking by a Robot Using an Online Continuous Model to Generate Turn-taking Cues
ICMI '19: 2019 International Conference on Multimodal InteractionTurn-taking in human-robot interaction is a crucial part of spoken dialogue systems, but current models do not allow for human-like turn-taking speed seen in natural conversation. In this work we propose combining two independent prediction models. A ...
Timing in multimodal turn-taking interactions: control and analysis using timed Petri nets
Inaugural Special Issue: Intersection of Systems Sciences and Human SciencesTurn-taking interactions with humans are multimodal and reciprocal in nature. In addition, the timing of actions is of great importance, as it influences both social and task strategies. To enable the precise control and analysis of timed discrete ...
Grounding and turn-taking in multimodal multiparty conversation
HCI'13: Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IVThis study explores the empirical basis for multimodal conversation control acts. Applying conversation analysis as an exploratory approach, we attempt to illuminate the control functions of paralinguistic behaviors in managing multiparty conversation. ...
Comments