Vinoth Pandian Sermuga Pandian
Sarah Suleri
Matthias Jarke
ABSTRACT
UI designers look for inspirational examples from existing UI designs during the prototyping process. However, they have to reconstruct these example UI designs from scratch to edit content or apply styling. The existing solution attempts to make UI screens into editable vector graphics using image segmentation techniques. In this research, we aim to use deep learning and gestalt laws-based algorithms to convert UI screens to editable blueprints by identifying the constituent UI element categories, their location, dimension, text content, and layout hierarchy. In this paper, we present a proof-of-concept web application that uses the UI screens and annotations from the RICO dataset and generates an editable blueprint vector graphic, and a UI layout tree. With this research, we aim to support UX designers in reconstructing UI screens and communicating UI layout information to developers.
- Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. 2017. Rico: A mobile app dataset for building data-driven design applications. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM, 845--854.Google Scholar
Digital Library
- Scarlett R Herring, Chia Chen Chang, Jesse Krantzler, and Brian P Bailey. 2009. Getting inspired! Understanding how and why examples are used in creative design practice. In Conference on Human Factors in Computing Systems - Proceedings. 87--96. https://doi.org/10.1145/1518701.1518717Google ScholarDigital Library
- Sarah Suleri, Nilda Kipi, Linh Chi Tran, and Matthias Jarke. 2019. UI Design Pattern-driven Rapid Prototyping for Agile Development of Mobile Applications (MobileHCI '19). ACM, New York, NY, USA, Article 52, 6 pages. https://doi.org/10.1145/3338286.3344399Google ScholarDigital Library
- Sarah Suleri, Vinoth Pandian Sermuga Pandian, Svetlana Shishkovets, and Matthias Jarke. 2019. Eve: A Sketch-based Software Prototyping Workbench (CHIEA '19). ACM, New York, NY, USA, Article LBW1410, 6 pages. https://doi.org/10.1145/3290607.3312994Google ScholarDigital Library
- Amanda Swearngin, Mira Dontcheva, Wilmot Li, Joel Brandt, Morgan Dixon, Andrew J Ko, Adobe Research, and Paul G Allen School. 2018. Rewire: Interface Design Assistance from Examples. (2018). https://doi.org/10.1145/3173574.3174078Google ScholarDigital Library
Index Terms
- Blu: What GUIs are made of
Recommendations
Object detection for graphical user interface: old fashioned or deep learning or a combination?
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringDetecting Graphical User Interface (GUI) elements in GUI images is a domain-specific object detection task. It supports many software engineering tasks, such as GUI animation and testing, GUI search and code generation. Existing studies for GUI element ...
Utilizing Self-Expression Template Method in User Interface Design - Three Design Cases
AcademicMindTrek '13: Proceedings of International Conference on Making Sense of Converging MediaIn this paper, we introduce our Self-Expression Template Method for design. We also briefly present three different UI design cases utilizing the template with authentic users and UI professionals from both research and industry. This method provided ...
Impact of using UI Design Patterns on the Workload of Rapid Prototyping of Smartphone Applications: An Experimental Study
MobileHCI '20: 22nd International Conference on Human-Computer Interaction with Mobile Devices and ServicesUI design pattern-driven prototyping introduces the idea of utilizing UI design patterns to create rapid prototypes. In this paper, we investigated the impact of using UI design pattern-driven approach on the subjective workload of rapid prototyping. ...
Comments