Hrvoje Benko
Andrew D. Wilson
ABSTRACT
This paper describes an interactive immersive experience using mid-air gestures to interact with a large curved display: a projected dome. Our Pinch-the-Sky Dome is an immersive installation where several users can interact simultaneously with omnidirectional data using freehand gestures. The system consists of a single centrally-located omnidirectional projector-camera unit where the projector is able to project an image spanning the entire 360 degrees and a camera is used to track gestures for navigation of the content. We combine speech commands with freehand pinch and clasping gestures and infra-red laser pointers to provide a highly immersive and interactive experience to several users inside the dome, with a very wide field of view for each user. The interactive applications include: 1) the astronomical data exploration, 2) social networking 3D graph visualizations, 3) immersive panoramic images, 4) 360 degree video conferencing, 5) a drawing canvas, and 6) a multi-user interactive game. Finally, we discuss the user reactions and feedback from two demo events where more than 1000 people had the chance to experience our work.
Supplemental Material
- Benko, H. 2009. Beyond Flat Surface Computing: Challenges of Depth-Aware and Curved Interfaces. In Proc. ACM MultiMedia '09. p. 935--944. Google Scholar
Digital Library
- Benko, H. and Wilson, A. D. 2010. Pinch-the-sky dome: freehand multi-point interactions with immersive omni-directional data. Extended Abstracts ACM SIGCHI '10. p. 3045--3050. Google ScholarDigital Library
- Benko, H., Wilson, A., and Balakrishnan, R. 2008. Sphere: Multi-Touch Interactions on a Spherical Display. In Proc. ACM UIST '08. p. 77--86. Google ScholarDigital Library
- Buxton, W. 1990. A three-state model of graphical input. In Proc. IFIP Tc13 Third International Conference on Human-Computer interaction (August 27--31, 1990). p. 449--456. Google ScholarDigital Library
- Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. 1993. Surround-screen projection-based virtual reality: The design and implementation of the CAVE. In Proc. ACM SIGGRAPH '93. p. 135--142. Google ScholarDigital Library
- Emmart, C. 2001. Tools and Techniques for Realtime Dome Production and Education. Computer Graphics for Large Scale Immersive Theaters. SIGGRAPH '01 Course Notes.Google Scholar
- Fitzmaurice, G., Khan, A., Buxton, W., Kurtenbach, G., and Balakrishnan, R. 2003. Sentient data access via a diverse society of devices. ACM Queue. p. 53--62. Google ScholarDigital Library
- Fukuchi, K., Sato, T., Mamiya, H., and Koike, H. 2010. Pac-pac: pinching gesture recognition for tabletop entertainment system. In Proc. ACM AVI '10. p. 267--273. Google ScholarDigital Library
- Gaitatzes, A., Papaioannou, G., Christopoulos, D., and Zyba, G. 2006. Media productions for a dome display system. In Proc. ACM VRST '06. p. 261--264. Google ScholarDigital Library
- Gray, J. and Szalay, A. 2002. The World Wide Telescope: An Archetype for Online Science. Microsoft Research Technical Report MSR-TR-2002-75. June '02.Google Scholar
- Hilliges, O., Izadi, S., Wilson, A. D., Hodges, S., Garcia-Mendoza, A., and Butz, A. 2009. Interactions in the Air: Adding Further Depth to Interactive Tabletops. In Proc. ACM UIST '09. p. 139--148. Google ScholarDigital Library
- Hua, H., Brown, L. D., and Gao, C. 2004. Scape: Supporting Stereoscopic Collaboration in Augmented and Projective Environments. IEEE Computer Graphics and Applications. 24, 1 (Jan. 2004). p. 66--75. Google ScholarDigital Library
- Kaiser, E., Olwal, A., McGee, D., Benko, H., Corradini, A., Li, X., Cohen, P., and Feiner, S. 2003. Mutual Disambiguation of 3D Multimodal Interaction in Augmented and Virtual Reality. In Proc. ICMI '03. p. 12--19. Google ScholarDigital Library
- Krueger, M. W., Gionfriddo, T., and Hinrichsen, K. 1985. VIDEOPLACE---an artificial reality. SIGCHI Bull. 16, 4 (Apr. '85). p. 35--40. Google ScholarDigital Library
- LaViola, J. 2000. MSVT: A Virtual Reality-Based Multimodal Scientific Visualization Tool. In Proc. IASTED International Conference on Computer Graphics and Imaging. p. 1--7.Google Scholar
- Magnor, M., Sen, P., Kniss, J., Angel, E., and Wenger, S. 2010. Progress in Rendering and Modeling for Digital Planetariums. In Proc. of EUROGRAPHICS '10.Google Scholar
- Matsushita, N. and Rekimoto, J. 1997. HoloWall: Designing a Finger, Hand, Body, and Object Sensitive Wall. In Proc. ACM UIST '99. p. 209--210. Google ScholarDigital Library
- Mapes, D. P. and Moshell, J. M. 1995. A two-handed interface for object manipulation in virtual environments. PRESENCE: Teleoperators and Virtual Environments, 4(4). p. 403--416.Google Scholar
- Myers, B. A., Bhatnagar, R., Nichols, J., Peck, C. H., Kong, D., Miller, R., and Long, A. C. 2002. Interacting at a distance: measuring the performance of laser pointers and other devices. In Proc. ACM SIGCHI '02. p. 33--40. Google ScholarDigital Library
- Oh, J.-Y. and Stuerzlinger, W. 2002. Laser Pointers as Collaborative Pointing Devices, In Proc. Graphics Interface '02. p. 141--149.Google Scholar
- Olsen, D. R. and Nielsen, T. 2001. Laser pointer Interaction. In Proc. ACM SIGCHI '01. p. 17--22. Google ScholarDigital Library
- Pinhanez, C. S. 2001. The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. In Proc. UBICOMP '01. p. 315--331. Google ScholarDigital Library
- Poupyrev, I., Billinghurst, M., Weghorst, S., and Ichikawa, T. 1996. The go-go interaction technique: non-linear mapping for direct manipulation in VR. In Proc. ACM UIST '96. p. 79--80. Google ScholarDigital Library
- Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. 1998. The Office of the Future: A Unified Approach to Image-Based Modeling and Spatially Immersive Displays. In Proc. ACM SIGGRAPH '98. p. 179--188. Google ScholarDigital Library
- Sato, Y., Saito, M. and Koike, H. 2001. Real-time input of 3D pose and gestures of a user's hand and its applications for HCI. In. Proc. IEEE VR '01. p. 79--86. Google ScholarDigital Library
- Shoemaker, G., Tang, A., and Booth, K. S. 2007. Shadow reaching: a new perspective on interaction for large displays. In Proc. ACM UIST '07. p. 53--56. Google ScholarDigital Library
- Simon, A. 2005. First-person experience and usability of co-located interaction in a projection-based virtual environment. In Proc. ACM VRST '05. p. 23--30. Google ScholarDigital Library
- Simon, A. and Göbel, M. 2002. The i-Cone -- A Panoramic Display System for Virtual Environments. In Proc. Pacific Conference on Computer Graphics and Applications. p. 3--7. Google ScholarDigital Library
- Stanney, K. M., Mourant, R. R., and Kennedy, R. S. 1998. Human Factors Issues in Virtual Environments: A Review of the Literature. Presence 7(4). p. 327--351. Google ScholarDigital Library
- Wilson, A. 2006. Robust Computer Vision-Based Detection of Pinching for One and Two-Handed Gesture Input. In Proc. ACM UIST '06. p. 255--258. Google ScholarDigital Library
- Wilson, A. 2007. Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction. In Proc. IEEE TABLETOP '07. p. 201--204.Google ScholarCross Ref
- Wren, C., Azarbayejani, A., Darrell, T., and Pentland, A. 1997. Pfinder: real-time tracking of the human body. IEEE Trans. PAMI, 19 (7). p. 780--785. Google ScholarDigital Library
- Yilmaz, A., Javed, O., and Shah, M. 2006. Object tracking: A survey. ACM Computing Surveys. 38(4). (Dec. '06), Article #13. Google ScholarDigital Library
Index Terms
- Multi-point interactions with immersive omnidirectional visualizations in a dome
Recommendations
Pinch-the-sky dome: freehand multi-point interactions with immersive omni-directional data
CHI EA '10: CHI '10 Extended Abstracts on Human Factors in Computing SystemsPinch-the-Sky Dome is a large immersive installation where several users can interact simultaneously with omni-directional data inside of a tilted geodesic dome. Our system consists of an omni-directional projector-camera unit in the center of the dome. ...
Vulture: a mid-air word-gesture keyboard
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWord-gesture keyboards enable fast text entry by letting users draw the shape of a word on the input surface. Such keyboards have been used extensively for touch devices, but not in mid-air, even though their fluent gestural input seems well suited for ...
Interactive dome experiences: designing astrosurf
AcademicMindtrek '16: Proceedings of the 20th International Academic Mindtrek ConferenceDome and display technology have evolved to the point that designing interactive dome experiences no longer requires expertise with dome technology. In particular, the DomeLab provides an interface for designers to work with popular game development ...
Comments