Marcos Serrano
David Juras
Laurence Nigay
ABSTRACT
In this paper we address the problem of the development of multimodal interfaces. We describe a three-dimensional characterization space for software components along with its implementation in a component-based platform for rapidly developing multimodal interfaces. By graphically assembling components, the designer/developer describes the transformation chain from physical devices to tasks and vice-versa. In this context, the key point is to identify generic components that can be reused for different multimodal applications. Nevertheless for flexibility purposes, a mixed approach that enables the designer to use both generic components and tailored components is required. As a consequence, our characterization space includes one axis dedicated to the reusability aspect of a component. The two other axes of our characterization space, respectively depict the role of the component in the data-flow from devices to tasks and the level of specification of the component. We illustrate our three dimensional characterization space as well as the implemented tool based on it using a multimodal map navigator.
- Ballagas, R., Ringel, M., Stone, M. and Borchers, J. (2003). iStuff: a physical user interface toolkit for ubiquitous computing environments. Proc. of Human factors in computing systems 2003, ACM Press, pp. 537--544. Google Scholar
Digital Library
- Benoit, A., Bonnaud, L. Caplier, A., Damousis, L., Tzovaras, D., Jourde, F., Nigay, L., Serrano, M., and Lawson, J-Y. (2006). Multimodal signal processing and interaction for a driving simulation: component--based architecture. Journal on Multimodal User Interfaces, 1, 1, pp. 49--58.Google ScholarCross Ref
- Bolt, R. A. (1980). Put-that-there: voice and gesture at the graphics interface. SIGGRAPH'80, 14, 3, pp. 262--270. Google ScholarDigital Library
- Bouchet, J., Nigay, L. (2004). ICARE software components for rapidly developing multimodal interfaces. In Proceedings of ICMI 2004, State College, USA, pp. 251--258. Google ScholarDigital Library
- Buxton, W. (1983). Lexical and pragmatic considerations of input structures. ACM SIGGRAPH CG, 17(1), pp. 31--37. Google ScholarDigital Library
- Card, S., MacKinlay, J., and Robertson, G. (1991). A morphological analysis of the design space of input devices. ACM Transactions on Information Systems, 9, pp. 99--122. Google ScholarDigital Library
- Cohen, P., McGee, D., Clow, J. (2000). The efficiency of multimodal interaction for a map-based task. Proc. of the 6th ANLP Conference, ACM Press, pp. 331--338. Google ScholarDigital Library
- COM. www.microsoft.com/comGoogle Scholar
- Dietz, P. and Leigh, D. (2001). DiamondTouch: a multi-user touch technology. Proc. of UIST'01, ACM Press, pp. 219--226. Google ScholarDigital Library
- Dragicevic, P., and Fekete, J. D. (2001). Input device selection and interaction configuration with ICON. Joint Proc. of IHM'01 and HCI'01, Springer Verlag, pp. 543--558.Google ScholarCross Ref
- EJB. http://java.sun.com/products/ejb/.Google Scholar
- Foley, J., Wallace, V.L. and Chan, P. (1984). The human factors of graphics interaction techniques. IEEE Computer Graphics and Applications, 11, pp. 13--48. Google ScholarDigital Library
- Gray, P., Ramsay, A., Serrano, M. (2007). A demonstration of the OpenInterface Interaction Development Environment. UIST'07 Adjunct Proc., ACM Press, pp. 39--40.Google Scholar
- Greenberg, S. and Fitchett, C. (2001). Phidgets: easy development of physical interfaces throught physical widgets. In Proc. of UIST'01, ACM Press, pp. 209--218. Google ScholarDigital Library
- Heer, J., Card, S., Landay, J. (2005). Prefuse: a toolkit for interactive information visualization. Proc. of the SIGCHI conference on Human factors in computing systems, ACM press, pp. 421--430. Google ScholarDigital Library
- Hinckley, K., and Sinclair, M. (1999). Touch-sensing input devices. ACM. Proc. of CHI'99, ACM Press, pp. 223--230. Google ScholarDigital Library
- Hudson, S., Mankoff, J. (2006). Rapid construction of functioning physical interfaces from cardboard, thumbtacks, tin foil and masking tape. In Proc. of UIST'06, ACM Press, pp. 289--298. Google ScholarDigital Library
- Huot, S., Dumas, C., Dragicevic, P, Fekete, J. and Hégron, G. (2004). The MaggLite post-WIMP toolkit: draw it, connect it and run it. In Proc. of UIST'04, ACM Press, pp. 257--266. Google ScholarDigital Library
- Interface-Z, www.interface-z.com.Google Scholar
- iPhone, www.apple.com/iphone.Google Scholar
- Letessier, J., Berard, F. (2004). Visual tracking of bare fingers for interactive surfaces. In Proc. of UIST'04, ACM Press, pp. 119--122. Google ScholarDigital Library
- Lingrand, D. and Riveill, M. (2006). Input interactions and context component based modelisations: differences and similarities. Proc. of AVI'06. ACM Press. pp. 19--22. Google ScholarDigital Library
- Maynes-Aminzade, D., Winograd, T., and Igarashi, T. (2007). Eyepatch: prototyping camera-based interaction through Examples. In Proc. UIST'07, ACM Press, pp. 33--42. Google ScholarDigital Library
- Morch, A. I., et al. (2004). Component-based technologies for end--user development. Comunications of the ACM, Volume 47, Issue 9, pp. 59--62. Google ScholarDigital Library
- Multitel. www.multitel.beGoogle Scholar
- Myers, B., Hudson, S. E., Pausch, R. (2000). Past, present, and future of user interface software tools. Transactions on Computer-Human Interaction, Vol. 7, No. 1, pp. 3--28. Google ScholarDigital Library
- Nigay, L., Coutaz, J. (1997). Multifeature systems: the CARE properties and their impact on software design. intelligence and multimodality in multimedia interfaces, AAAI Press.Google Scholar
- Olsen, D. R. Jr. (2007). Evaluating user interface systems research. Proc. of UIST'07, ACM Press, pp. 251--258. Google ScholarDigital Library
- OMG -- CORBA. www.corba.orgGoogle Scholar
- OpenInterface European project. IST Framework 6 STREP funded by the European Commission (FP6--35182). www.oi-project.org.Google Scholar
- OSGi. www.osgi.orgGoogle Scholar
- Ray, A., Bowman, D. A., (2007). Towards a system for reusable 3D interaction techniques. Proc. Of the 2007 ACM symposium on Virtual reality software and technology, ACM Press, pp. 187--190. Google ScholarDigital Library
- Shen, C., Vernier, F., Forlines, C., and Ringel, M. (2004). DiamondSpin: an extensible toolkit for around-the-table interaction. Proc. of CHI'04, ACM Press, pp. 167--174. Google ScholarDigital Library
- Shirai, A., Geslin, E., Richir, S. (2007). WiiMedia: motion analysis methods and applications using a consumer video game controller. Proc. of the SIGGRAPH Symposium on Video Games, ACM Press, pp. 133--140. Google ScholarDigital Library
- Sinha, A., Landay, J. (2003). Capturing user tests in a multimodal, multidevice informal prototyping tool. Proc. of ICMI'03, ACM Press, pp. 117--124. Google ScholarDigital Library
- Sutton, S., Cole, R. (1997). The CSLU toolkit: rapid prototyping of spoken language systems. Proc. of UIST'97, ACM Press, pp. 85--86. Google ScholarDigital Library
- The UIMS tool developers workshop, A metamodel for the runtime architecture of an interactive system. SIGCHI Bulletin (1992), pp. 32--37. Google ScholarDigital Library
- Tse, E., Shen, C., Greenberg, S. and Forlines, C. (2006). Enabling interaction with single user applications through speech and gestures on a multi-user tabletop. Proc. of AVI'06, ACM Press, pp. 336--343. Google ScholarDigital Library
- Yahoo! Pipes. http://pipes.yahoo.comGoogle Scholar
Index Terms
- A three-dimensional characterization space of software components for rapidly developing multimodal interfaces
Recommendations
Temporal aspects of CARE-based multimodal fusion: from a fusion mechanism to composition components and WoZ components
ICMI-MLMI '09: Proceedings of the 2009 international conference on Multimodal interfacesThe CARE properties (Complementarity, Assignment, Redundancy and Equivalence) define various forms that multimodal input interaction can take. While Equivalence and Assignment express the availability and respective absence of choice between multiple ...
ICARE software components for rapidly developing multimodal interfaces
ICMI '04: Proceedings of the 6th international conference on Multimodal interfacesAlthough several real multimodal systems have been built, their development still remains a difficult task. In this paper we address this problem of development of multimodal interfaces by describing a component-based approach, called ICARE, for rapidly ...
OpenWizard: une approche pour la création et l'évaluation rapide de prototypes multimodaux
IHM '09: Proceedings of the 21st International Conference on Association Francophone d'Interaction Homme-MachineWe present OpenWizard, a wizard of oz component-based approach for rapidly prototyping and testing multimodal applications. OpenWizard allows the designer and the developer to rapidly test a non-fully functional multimodal prototype by replacing one ...
Comments