Simon Mayer
Friedemann Mattern
Skip Abstract Section
Abstract
With ever more everyday objects becoming “smart” due to embedded processors and communication capabilities, the provisioning of intuitive user interfaces to control smart things is quickly gaining importance. We present a model-based interface description scheme that enables automatic, modality-independent user interface generation. User interface description languages based on our approach carry enough information to suggest intuitive interfaces while still being easily producible for developers. This is enabled by describing the atomic interactive components of a device and capturing the semantics of interactions with the device. We propose a taxonomy of abstract sensing and actuation primitives and present a smartphone application that can act as a ubiquitous device controller. An evaluation of the mobile application in a laboratory setup, home environments, and an educational setting as well as the results of a user study highlight the accessibility of the proposed scheme for application developers and its suitability for controlling smart devices.
- Michel Beaudouin-Lafon. 2004. Designing Interaction, not Interfaces. In Proceedings of the Working Conference on Advanced Visual Interfaces (Gallipoli, Italy, May 2004), Maria Francesca Costabile (Ed.). ACM, New York, NY, 15--22. Google Scholar
- Michael Beigl, Albrecht Schmidt, Markus Lauff, and Hans-Werner Gellersen. 1998. The UbicompBrowser. In Proceedings of the 4th ERCIM Workshop on User Interfaces for All (Stockholm, Sweden, October 1998), Constantine Stephanidis and Annika Waern (Eds.). 51--86.Google Scholar
- Gaëlle Calvary, Joëlle Coutaz, David Thevenin, Quentin Limbourg, Laurent Bouillon, and Jean Vanderdonckt. 2003. A Unifying Reference Framework for Multi-Target User Interfaces. Interacting with Computers 15, 3 (2003), 289--308.Google Scholar
- P. M. Corcoran and J. Desbonnet. 1997. Browser-style Interfaces to a Home Automation Network. IEEE Transactions on Consumer Electronics 43, 4 (1997), 1063--1069. Google Scholar
- Roy T. Fielding. 2000. Architectural Styles and the Design of Network-based Software Architectures. Ph.D. Dissertation. University of California, Irvine. Google Scholar
- Krzysztof Z. Gajos, Daniel S. Weld, and Jacob O. Wobbrock. 2010. Automatically Generating Personalized User Interfaces with SUPPLE. Artificial Intelligence 174, 12/13 (2010), 910--950. DOI: http://dx.doi.org/doi:10.1016/j.artint.2010.05.005 Google Scholar
- Dominique Guinard, Vlad Trifa, Friedemann Mattern, and Erik Wilde. 2011. From the Internet of Things to the Web of Things: Resource Oriented Architecture and Best Practices. In Architecting the Internet of Things, Dieter Uckelmann, Mark Harrison, and Florian Michahelles (Eds.). Springer, Berlin, 97--129.Google Scholar
- James Helms, Robbie Schaefer, Kris Luyten, Jo Vermeulen, Marc Abrams, Adrien Coyette, and Jean Vanderdonckt. 2009. Human-Centered Engineering of Interactive Systems with the User Interface Markup Language. In Human-Centered Software Engineering—Software Engineering Models, Patterns and Architectures for HCI, Ahmed Seffah, Jean Vanderdonckt, and Michel C. Desmarais (Eds.). Springer, Berlin, Germany, 139--171.Google Scholar
- Todd D. Hodes, Randy H. Katz, Edouard Servan-Schreiber, and Lawrence Rowe. 1997. Composable Ad-hoc Mobile Services for Universal Interaction. In Proceedings of the 3rd Annual ACM/IEEE International Conference on Mobile Networking and Computing (Budapest, Hungary, September 1997), László Pap, Karen Sohraby, David B. Johnson, and Christopher Rose (Eds.). ACM, New York, NY, 1--12. Google Scholar
- Won Chul Kim and James D. Foley. 1993. Providing High-level Control and Expert Assistance in the User Interface Presentation Design. In Proceedings of the Human-Computer Interaction, INTERACT ’93, IFIP TC13 International Conference on Human-Computer Interaction (Amsterdam, The Netherlands, April 1993), Stacey Ashlund, Kevin Mullet, Austin Henderson, Erik Hollnagel, and Ted N. White (Eds.). ACM, New York, NY, 430--437. Google Scholar
- Andreas Lorenz. 2013. Architectural patterns for applications with external user interface elements. Pervasive and Mobile Computing 9, 2 (2013), 269--280. DOI: http://dx.doi.org/10.1016/j.pmcj.2012.09.006 Google Scholar
- Kieran Mathieson, Eileen Peacock, and Wynne W. Chin. 2001. Extending the Technology Acceptance Model: The Influence of Perceived User Resources. ACM SIGMIS Database 32, 3 (2001), 86--112. DOI: http://dx.doi.org/10.1145/506724.506730 Google Scholar
- Friedemann Mattern and Christian Floerkemeier. 2010. From the Internet of Computers to the Internet of Things. In From Active Data Management to Event-Based Systems and More, Kai Sachs, Ilia Petrov, and Pablo Guerrero (Eds.). LNCS, Vol. 6462. Springer, Berlin, 242--259. Google Scholar
- Simon Mayer and Dominique Guinard. 2011. An Extensible Discovery Service for Smart Things. In Proceedings of the 2nd International Workshop on the Web of Things (San Francisco, CA, June 2011), Dominique Guinard, Vlad Trifa, and Erik Wilde (Eds.). ACM, New York, NY. Google Scholar
- Simon Mayer, Dominique Guinard, and Vlad Trifa. 2010. Facilitating the Integration and Interaction of Real-World Services for the Web of Things. In UrbanIOT 2010; Workshop at the Internet of Things 2010 Conference (IoT 2010).Google Scholar
- Gerrit Meixner, Fabio Paternò, and Jean Vanderdonckt. 2011. Past, Present, and Future of Model-Based User Interface Development. i-com 10, 3 (2011), 2--11.Google Scholar
- Alan Messer, Anugeetha Kunjithapatham, Mithun Sheshagiri, Henry Song, Praveen Kumar, Phuong Nguyen, and Kyoung Hoon Yi. 2006. InterPlay: A Middleware for Seamless Device Integration and Task Orchestration in a Networked Home. In Proceedings of the 4th IEEE International Conference on Pervasive Computing and Communications (Pisa, Italy, March 2006). IEEE Computer Society, Washington, DC, 296--307. DOI: http://dx.doi.org/10.1109/PERCOM.2006.30 Google Scholar
- Brad Myers, Scott E. Hudson, and Randy Pausch. 2000. Past, Present, and Future of User Interface Software Tools. ACM Transactions on Computer-Human Interaction 7, 1 (2000), 3--28. DOI: http://dx.doi.org/10.1145/344949.344959 Google Scholar
- Brad A. Myers. 1990. A New Model for Handling Input. ACM Transactions on Information Systems 8, 3 (1990), 289--320. DOI: http://dx.doi.org/10.1145/98188.98204 Google Scholar
- David Navarre, Philippe Palanque, Jean-Francois Ladry, and Eric Barboni. 2009. ICOs: A Model-Based User Interface Description Technique Dedicated to Interactive Systems Addressing Usability, Reliability and Scalability. ACM Transactions on Computer-Human Interaction 16, 4 (2009), Article 18, 56 pages. DOI: http://dx.doi.org/10.1145/1614390.1614393 Google Scholar
- Jeffrey Nichols and Brad A. Myers. 2009. Creating a Lightweight User Interface Description Language: An Overview and Analysis of the Personal Universal Controller Project. ACM Transactions on Computer-Human Interaction 16, 4 (2009), Article 17, 37 pages. DOI: http://dx.doi.org/10.1145/1614390.1614392 Google Scholar
- Jeffrey Nichols, Brad A. Myers, Michael Higgins, Joseph Hughes, Thomas K. Harris, Roni Rosenfeld, and Mathilde Pignol. 2002. Generating Remote Control Interfaces for Complex Appliances. In Proceedings of the 2002 International Conference on Intelligent User Interfaces (San Francisco, CA, January 2002). ACM, New York, NY, 161--170. DOI: http://dx.doi.org/10.1145/571985.572008 Google Scholar
- Jeffrey Nichols, Brad A. Myers, and Kevin Litwack. 2004. Improving Automatic Interface Generation with Smart Templates. In Proceedings of the 2004 International Conference on Intelligent User Interfaces (Funchal, Portugal, January 2004), Jean Vanderdonckt, Nuno Jardim Nunes, and Charles Rich (Eds.). ACM, New York, NY, 286--288. DOI: http://dx.doi.org/10.1145/964442.964507 Google Scholar
- Dan R. Olsen, Sean Jefferies, Travis Nielsen, William Moyes, and Paul Fredrickson. 2000. Cross-modal Interaction using XWeb. In Proceedings of the 2000 International Conference on Intelligent User Interfaces (New Orleans, LA, USA, January 2000). ACM, New York, NY, 191--200. Google Scholar
- Benedikt Ostermaier, Matthias Kovatsch, and Silvia Santini. 2011. Connecting Things to the Web using Programmable Low-power WiFi Modules. In Proceedings of the 2nd International Workshop on the Web of Things (San Francisco, CA, USA, June 2011), Dominique Guinard, Vlad Trifa, and Erik Wilde (Eds.). ACM, New York, NY. Google Scholar
- Fabio Paternò. 2005. Model-based Tools for Pervasive Usability. Interacting with Computers 17, 3 (2005), 291--315.Google Scholar
- Fabio Paternò, Carmen Santoro, and Lucio Davide Spano. 2009. MARIA: A Universal, Declarative, Multiple Abstraction-Level Language for Service-Oriented Applications in Ubiquitous Environments. ACM Transactions on Computer-Human Interaction 16, 4 (2009), Article 19, 30 pages. DOI: http://dx.doi.org/10.1145/1614390.1614394 Google Scholar
- Shankar R. Ponnekanti, Brian Lee, Armando Fox, Pat Hanrahan, and Terry Winograd. 2001. ICrafter: A Service Framework for Ubiquitous Computing Environments. In Proceedings of the 3rd International Conference on Ubiquitous Computing (Atlanta, GA, September 2001) LNCS, Gregory D. Abowd, Barry Brumitt, and Steven A. Shafer (Eds.), Vol. 2201. Springer, Berlin, 56--75. Google Scholar
- Eiji Tokunaga, Hiroaki Kimura, Nobuyuki Kobayashi, and Tatsuo Nakajima. 2005. Virtual Tangible Widgets: Seamless Universal Interaction with Personal Sensing Devices. In Proceedings of the 7th International Conference on Multimodal Interfaces (Trento, Italy, October 2005), Gianni Lazzari, Fabio Pianesi, James L. Crowley, Kenji Mase, and Sharon L. Oviatt (Eds.). ACM, New York, NY, 325--332. DOI: http://dx.doi.org/10.1145/1088463.1088518 Google Scholar
- Markus Weiss and Dominique Guinard. 2010. Increasing Energy Awareness Through Web-enabled Power Outlets. In Proceedings of the 9th International Conference on Mobile and Ubiquitous Multimedia (Limmassol, Cyprus, December 2010), Marios C. Angelides, Lambros Lambrinos, Michael Rohs, and Enrico Rukzio (Eds.). ACM, New York, NY. Google Scholar
- Gottfried Zimmermann, Gregg Vanderheiden, and Al Gilman. 2002. Prototype Implementations for a Universal Remote Console Specification. In Extended Abstracts of the 2002 Conference on Human Factors in Computing Systems (Minneapolis, MN, April 2002), Loren G. Terveen and Dennis R. Wixon (Eds.). ACM, 510--511. Google Scholar
Index Terms
- User interfaces for smart things -- A generative approach with semantic interaction descriptions
Recommendations
Multimodal user interfaces for smart environments: the multi-access service platform
AVI '08: Proceedings of the working conference on Advanced visual interfacesUser interface modeling is a well accepted approach to handle increasing user interface complexity. The approach presented in this paper utilizes user interface models at runtime to provide a basis for user interface distribution and synchronization. ...
A Novel Smart Object-Driven UI Generation Approach for Mobile Devices in the Internet of Things
SmartObjects '15: Proceedings of the 1st International Workshop on Experiences with the Design and Implementation of Smart ObjectsThe broad adoption of the Internet of Things (IoT) is linked to the possibility to discover and interact easily with objects in the surroundings of users. Because of their characteristics and large diffusion, mobile devices are perfect to connect the ...
Collapsible user interfaces for information retrieval agents
IUI '99: Proceedings of the 4th international conference on Intelligent user interfaces
Comments