Marco Blumendorf
Dirk Roscher
ABSTRACT
The availability of numerous networked interaction devices within smart environments makes the exploitation of these devices for innovative and more natural interaction possible. In our work we make use of TVs with remote controls, picture frames, mobile phones, touch screens, stereos and PCs to create multimodal user interfaces. The combination of the interaction capabilities of the different devices allows to achieve a more suitable interaction for a situation. Changing situations can then require the dynamic redistribution of the created interfaces and the alteration of the used modalities and devices to keep up the interaction. In this paper we describe our approach for dynamically (re-) distributing user interfaces at run-time. A distribution component is responsible for determining the devices for the interaction based on the (changing) environment situation and the user interface requirements. The component provides possibilities to the application developer and to the user to influence the distribution according to their needs. A user interface model describes the interaction and the modality relations according to the CARE properties (Complementarity, Assignment, Redundancy and Equivalency) and a context model gathers and provides information about the environment.
- Marc Abrams, Constantinos Phanouriou, Alan L. Batongbacal, Stephen M. Williams, and Jonathan E. Shuster. Uiml: An appliance-independent xml user interface language. In Computer Networks, 31(11--16):1695--1708, 1999. Google Scholar
Digital Library
- Nicolas Barralon, Joëlle Coutaz, and Christophe Lachenal. Coupling interaction resources and technical support. In HCI International 2007, pages 13--22, 2007. Google ScholarDigital Library
- Silvia Berti and Fabio Paternò. Migratory multimodal interfaces in multidevice environments. In ICMI '05: Proceedings of the 7th international conference on Multimodal interfaces, pages 92--99, 2005. Google ScholarDigital Library
- Marco Blumendorf, Sebastian Feuerstack, and Sahin Albayrak. Multimodal user interaction in smart environments: Delivering distributed user interfaces. In Constructing Ambient Intelligence: AmI 2007 Workshops. 2007.Google Scholar
- Marco Blumendorf, Grzegorz Lehmann, Sebastian Feuerstack, and Sahin Albayrak. Executable models for human-computer interaction. In Design, Specification, and Verification of Interactive Systems: 15th International Workshop, DSV-IS 2008, pages 238--251, 2008. Google ScholarDigital Library
- J. Bouchet, L. Nigay, and T. Ganille. Icare software components for rapidly developing multimodal interfaces. In ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces, pages 251--258, 2004. Google ScholarDigital Library
- Tim Clerckx, Chris Vandervelpen, Kris Luyten, and Karin Coninx. A task-driven user interface architecture for ambient intelligent environments. In IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces, pages 309--311, 2006. Google ScholarDigital Library
- Joëlle Coutaz, Laurence Nigay, Daniel Salber, Ann Blandford, Jon May, and Richard M. Young. Four easy pieces for assessing the usability of multimodal interaction: The care properties. In INTERACT, pages 115--120, 1995.Google ScholarCross Ref
- Christian Elting and Michael Hellenschmidt. Strategies for self-organization and multimodal output coordination in distributed device environments. In Workshop on Artificial Intelligence in Mobile Systems 2004 In conjunction with UbiComp 2004, 2004.Google Scholar
- Meriam Horchani, Benjamin Caron, Laurence Nigay, and Franck Panaget. Natural multimodal dialogue systems: a configurable dialogue and presentation strategies component. In ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces, pages 291--298, 2007. Google ScholarDigital Library
- Thomas Kirste. Dynamite - dynamisch adaptive multimodale it ensembles. In Tagungsband: Forschungsoffensive "Software Engineering 2006", 2004.Google Scholar
- Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte, Laurent Bouillon, and Víctor López-Jaquero. Usixml: A language supporting multi-path development of user interfaces. In EHCI/DS-VIS, volume 3425 of Lecture Notes in Computer Science, pages 200--220. Springer, 2004. Google ScholarDigital Library
- Kris Luyten and Karin Coninx. Distributed user interface elements to support smart interaction spaces. In ISM '05: Proceedings of the Seventh IEEE International Symposium on Multimedia, pages 277--286, 2005. Google ScholarDigital Library
- Kris Luyten, Chris Vandervelpen, and Karin Coninx. Migratable user interface descriptions in component-based development. In DSV-IS, pages 44--58, 2002. Google ScholarDigital Library
- Fabio Paterno, Carmen Santoro, Jani Mantyjarvi, Giulio Mori, and Sandro Sansone. Authoring pervasive multimodal user interfaces. In International Journal of Web Engineering and Technology, 4:235--261(27), 2008. Google ScholarDigital Library
- Dirk Roscher, Marco Blumendorf, and Sahin Albayrak. Using meta user interfaces to control multimodal interaction in smart environments. In Proceedings of the IUI'09 Workshop on Model Driven Development of Advanced User Interfaces, Volume 439 of CEUR Workshop Proceedings, 2009. Google ScholarDigital Library
- Chris Vandervelpen, Geert Vanderhulst, Kris Luyten, and Karin Coninx. Light-weight distributed web interfaces: Preparing the web for heterogeneous environments. In ICWE, pages 197--202, 2005. Google ScholarDigital Library
Index Terms
- Dynamic user interface distribution for flexible multimodal interaction
Recommendations
Multimodal user interfaces for smart environments: the multi-access service platform
AVI '08: Proceedings of the working conference on Advanced visual interfacesUser interface modeling is a well accepted approach to handle increasing user interface complexity. The approach presented in this paper utilizes user interface models at runtime to provide a basis for user interface distribution and synchronization. ...
A meta user interface to control multimodal interaction in smart environments
IUI '09: Proceedings of the 14th international conference on Intelligent user interfacesSmart environments bring together multiple users, (interaction) resources and services. This creates complex and unpredictable interactive computing environments that are hard to understand. Users thus have difficulties to build up their mental model of ...
Distributing User Interfaces
DUI '14: Proceedings of the 2014 Workshop on Distributed User Interfaces and Multimodal InteractionThe distribution of user interfaces is a reality. To represent this reality this paper presents a metamodel to characterize user interface distribution capabilities and states. This metamodel allows analyzers/designers to manipulate user interface ...
Comments