Sean Follmer
Daniel Leithinger
Alex Olwal
Hiroshi Ishii
ABSTRACT
Malleable and organic user interfaces have the potential to enable radically new forms of interactions and expressiveness through flexible, free-form and computationally controlled shapes and displays. This work, specifically focuses on particle jamming as a simple, effective method for flexible, shape-changing user interfaces where programmatic control of material stiffness enables haptic feedback, deformation, tunable affordances and control gain. We introduce a compact, low-power pneumatic jamming system suitable for mobile devices, and a new hydraulic-based technique with fast, silent actuation and optical shape sensing. We enable jamming structures to sense input and function as interaction devices through two contributed methods for high-resolution shape sensing using: 1) index-matched particles and fluids, and 2) capacitive and electric field sensing. We explore the design space of malleable and organic user interfaces enabled by jamming through four motivational prototypes that highlight jamming's potential in HCI, including applications for tabletops, tablets and for portable shape-changing mobile devices.
Supplemental Material
- Amend, J., Brown, E., Rodenberg, N., Jaeger, H., and Lipson, H. A Positive Pressure Universal Gripper Based on the Jamming of Granular Material. IEEE Transactions on Robotics (2012), 341--350.Google Scholar
- Benko, H. Beyond flat surface computing: challenges of depth-aware and curved interfaces. In ACM MM '09, ACM Press (New York, New York, USA, Oct. 2009), 935--944. Google ScholarDigital Library
- Brown, E., Rodenberg, N., Amend, J., Mozeika, A., Steltz, E., Zakin, M. R., Lipson, H., and Jaeger, H. M. Universal robotic gripper based on the jamming of granular material. Proceedings of the National Academy of Sciences 107, 44 (Oct. 2010), 18809--18814.Google ScholarCross Ref
- Cassinelli, A., and Ishikawa, M. Khronos projector. In ACM SIG-GRAPH 2005 Emerging Technologies, SIGGRAPH '05, ACM Press (New York, New York, USA, 2005). Google ScholarDigital Library
- Cates, M., Wittmer, J., Bouchaud, J. P., and Claudin, P. Jamming, Force Chains, and Fragile Matter. Physical Review Letters 81, 9 (Aug. 1998), 1841--1844.Google ScholarCross Ref
- Cheng, N., Lobovsky, M., Keating, S., Setapen, A., Gero, K. I., Hosoi, A., and Iagnemma, K. Design and Analysis of a Robust, Low-Cost, Highly Articulated Manipulator Enabled by Jamming of Granular Media. IEEE ICRA (2012), 4328--4333.Google Scholar
- Coelho, M., and Zigelbaum, J. Shape-changing interfaces. Personal and Ubiquitous Computing 15, 2 (July 2010), 161--173. Google ScholarDigital Library
- Follmer, S., Johnson, M., Adelson, E., and Ishii, H. deForm: an interactive malleable surface for capturing 2.5D arbitrary objects, tools and touch. In UIST '11, ACM Press (Oct. 2011), 527--536. Google ScholarDigital Library
- Gupta, S., Campbell, T., Hightower, J. R., and Patel, S. N. SqueezeBlock: using virtual springs in mobile devices for eyes-free interaction. In UIST '10, ACM Press (Oct. 2010), 101--104. Google ScholarDigital Library
- Harrison, C., and Hudson, S. E. Providing dynamically changeable physical buttons on a visual display. In CHI '09, ACM Press (Apr. 2009), 299--308. Google ScholarDigital Library
- Hemmert, F., Hamann, S., Löwe, M., Wohlauf, A., and Joost, G. Shape-changing mobiles: tapering in one-dimensional deformational displays in mobile phones. In TEI '10, ACM Press (Jan. 2010), 249-- 252. Google ScholarDigital Library
- Herkenrath, G., Karrer, T., and Borchers, J. Twend: twisting and bending as new interaction gesture in mobile devices. In CHI EA '08, ACM Press (New York, New York, USA, Apr. 2008), 3819--3824. Google ScholarDigital Library
- Hilliges, O., Kim, D., and Izadi, S. Creating malleable interactive surfaces using liquid displacement sensing. In Proc. Tabletop 2008, IEEE Press (Oct. 2008), 157--160.Google ScholarCross Ref
- Hook, J., Taylor, S., Butler, A., Villar, N., and Izadi, S. A reconfigurable ferromagnetic input device. In Proc. UIST 2009, ACM Press (New York, New York, USA, 2009), 51--54. Google ScholarDigital Library
- Huber, J. E., Fleck, N. a., and Ashby, M. F. The selection of mechanical actuators based on performance indices. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 453, 1965 (Oct. 1997), 2185--2205.Google Scholar
- Huijben, F., Herwijnen, F. V. A. N., and Nijsse, R. Vacuumatics 3D-Formwork Systems : Customized free-form solidification. Nano, figure 1 (2009), 1--4.Google Scholar
- Ishii, H., Ratti, C., Piper, B., Wang, Y., Biderman, A., and Ben-Joseph, E. Bringing Clay and Sand into Digital Design Continuous Tangible user Interfaces. BT Technology Journal 22, 4 (Oct. 2004), 287--299. Google ScholarDigital Library
- Iwata, H., Yano, H., Nakaizumi, F., and Kawamura, R. Project FEELEX: adding haptic surface to graphics. In SIGGRAPH '01, ACM Press (Aug. 2001), 469--476. Google ScholarDigital Library
- Jansen, Y., Karrer, T., and Borchers, J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. In Proc. ITS 2010, ACM Press (2010), 11--14. Google ScholarDigital Library
- Kim, S., Kim, H., Lee, B., Nam, T.-J., and Lee, W. Inflatable mouse: volume-adjustable mouse with air-pressure-sensitive input and haptic feedback. In CHI '08, ACM Press (Apr. 2008), 211--224. Google ScholarDigital Library
- Lahey, B., Girouard, A., Burleson, W., and Vertegaal, R. PaperPhone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In CHI '11, ACM Press (May 2011), 1303--1312. Google ScholarDigital Library
- Liu, A. Jamming is not just cool any more. Nature 396, November (1998), 21--22.Google Scholar
- Loeve, A. J., Ven, O. S., Vogel, J. G., Breedveld, P., and Dankelman, J. Vacuum packed particles as flexible endoscope guides with controllable rigidity. Granular Matter 12, 6 (June 2010), 543--554.Google ScholarCross Ref
- Marquardt, N., Nacenta, M. A., Young, J. E., Carpendale, S., Greenberg, S., and Sharlin, E. The Haptic Tabletop Puck: tactile feedback for interactive tabletops. In ITS '09, ACM Press (Nov. 2009), 85--92. Google ScholarDigital Library
- Matoba, Y., Sato, T., Takahashi, N., and Koike, H. Claytricsurface: an interactive surface with dynamic softness control capability. In SIGGRAPH '12 Emerging Technologies, ACM (2012), 6:1--6:1. Google ScholarDigital Library
- Mazzone, A., Spagno, C., and Kunz, A. The HoverMesh: a deformable structure based on vacuum cells: new advances in the research of tangible user interfaces. In ACE '04, ACM Press (Sept. 2004), 187--193. Google ScholarDigital Library
- Mitsuda, T., Kuge, S., Wakabayashi, M., and Kawamura, S. Wearable haptic display by the use of a Particle Mechanical Constraint. Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002 (2002), 153--158. Google ScholarDigital Library
- Overholt, D. The MATRIX: a novel controller for musical expression. In Proc. NIME 2001, NIME '01, AMC (Singapore, 2001), 1--4. Google ScholarDigital Library
- Park, Y.-L., Majidi, C., Kramer, R., Beérard, P., and Wood, R. J. Hyperelastic pressure sensing with a liquid-embedded elastomer. Journal of Micromechanics and Microengineering 20, 12 (Dec. 2010), 125029.Google ScholarCross Ref
- Parkes, A., Poupyrev, I., and Ishii, H. Designing kinetic interactions for organic user interfaces. Communications of the ACM 51, 6 (June 2008), 58--65. Google ScholarDigital Library
- Rekimoto, J. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proc. CHI 2002, CHI '02, ACM Press (New York, New York, USA, 2002), 113--120. Google ScholarDigital Library
- Schwesig, C., Poupyrev, I., and Mori, E. Gummi: a bendable computer. In Proceedings of the 2004 conference on Human factors in computing systems - CHI '04, ACM Press (New York, New York, USA, Apr. 2004), 263--270. Google ScholarDigital Library
- Sergio, M., Manaresi, N., and Tartagni, M. A Textile Based Capacitive Pressure Sensor. Sensors, 2002. (2002), 1625--1630.Google ScholarCross Ref
- Smith, R. T., Thomas, B. H., and Piekarski, W. Digital foam interaction techniques for 3D modeling. In Virtual Reality Software and Technology (2008), 61--68. Google ScholarDigital Library
- Steltz, E., Mozeika, A., and Rembisz, J. Jamming as an enabling technology for soft robotics. Electroactive Polymer Actuators and Devices (EAPAD), 764225.Google Scholar
- Steltz, E., Mozeika, A., Rodenberg, N., Brown, E., and Jaeger, H. JSEL: Jamming Skin Enabled Locomotion. 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (Oct. 2009), 5672--5677. Google ScholarDigital Library
- Taylor, B. T., and Bove, V. M. Graspables: grasp-recognition as a user interface. In Proceedings of the 27th international conference on Human factors in computing systems - CHI '09, ACM Press (New York, New York, USA, Apr. 2009), 917--926. Google ScholarDigital Library
- Torquato, S., Truskett, T., and Debenedetti, P. Is random close packing of spheres well defined? Physical review letters 84, 10 (Mar. 2000), 2064--7.Google Scholar
- Vertegaal, R., and Poupyrev, I. Organic User Interfaces. Communications of the ACM 51, 6 (June 2008), 26--30. Google ScholarDigital Library
- Vlack, K., Mizota, T., Kawakami, N., Kamiyama, K., Kajimoto, H., and Tachi, S. Gelforce: a vision-based traction field computer interface. In Ext. Abstracts CHI 2005, ACM Press (2005), 1154--1155. Google ScholarDigital Library
- White, Tom. Introducing Liquid Haptics in High Bandwidth Human Computer Interfaces. PhD thesis, MIT, 1998.Google Scholar
Index Terms
- Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices
Recommendations
Shape-changing interfaces: a review of the design space and open research questions
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsShape change is increasingly used in physical user interfaces, both as input and output. Yet, the progress made and the key research questions for shape-changing interfaces are rarely analyzed systematically. We review a sample of existing work on shape-...
SnowGlobe: a spherical fish-tank VR display
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing SystemsIn this paper, we present a spherical display with Fish-Tank VR as a means for interacting with three- dimensional objects. We implemented the spherical display by reflecting a projected image off a hemispherical mirror, allowing for a seamless ...
FlexView: an evaluation of depth navigation on deformable mobile devices
TEI '13: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied InteractionWe present FlexView, a set of interaction techniques for Z-axis navigation on touch-enabled flexible mobile devices. FlexView augments touch input with bend to navigate through depth-arranged content. To investigate Z-axis navigation with FlexView, we ...
Comments