ABSTRACT
Using the space above desktop input devices adds a rich new input channel to desktop interaction. Input in this elevated layer has been previously used to modify the granularity of a 2D slider, navigate layers of a 3D body scan above a multitouch table and access vertically stacked menus. However, designing these interactions is challenging because the lack of haptic and direct visual feedback easily leads to input errors. For bare finger input, the user's fingers needs to reliably enter and stay inside the interactive layer, and engagement techniques such as midair clicking have to be disambiguated from leaving the layer. These issues have been addressed for interactions in which users operate other devices in midair, but there is little guidance for the design of bare finger input in this space.
In this paper, we present the results of two user studies that inform the design of finger input above desktop devices. Our studies show that 2 cm is the minimum thickness of the above-surface volume that users can reliably remain within. We found that when accessing midair layers, users do not automatically move to the same height. To address this, we introduce a technique that dynamically determines the height at which the layer is placed, depending on the velocity profile of the user's initial finger movement into midair. Finally, we propose a technique that reliably distinguishes clicking from homing movements, based on the user's hand shape. We structure the presentation of our findings using Buxton's three-state input model, adding additional states and transitions for above-surface interactions.
Supplemental Material
- A. Banerjee, J. Burstyn, A. Girouard, and R. Vertegaal. Pointable: an in-air pointing technique to manipulate out-of-reach targets on tabletops. In ITS '11, 11--20. 2011. Google ScholarDigital Library
- W. Buxton. A three-state model of graphical input. In INTERACT '90, 449--456. 1990. Google ScholarDigital Library
- F. Camp, A. Schick, and R. Stiefelhagen. How to click in mid-air. In Distributed, Ambient, and Pervasive Interactions, volume 8028 of LCNS, 78--86. 2013.Google Scholar
- M. Desmurget, M. Jordan, C. Prablanc, and M. Jeannerod. Constrained and unconstrained movements involve different control strategies. J NEUROPHYSIOL, 77(3):1644--1650, 1997.Google ScholarCross Ref
- O. Hilliges, S. Izadi, A. D. Wilson, S. Hodges, A. Garcia-Mendoza, and A. Butz. Interactions in the air: adding further depth to interactive tabletops. In UIST '09, 139--148. 2009. Google ScholarDigital Library
- M. Jeannerod. The timing of natural prehension movements. J MOTOR BEHAV, 16(3):235, 1984.Google ScholarCross Ref
- H. Kato and H. Yanagihara. Pacman ui: vision-based finger detection for positioning and clicking manipulations. In MobileHCI '13, 464--467. 2013. Google ScholarDigital Library
- R. S. Kattinakere, T. Grossman, and S. Subramanian. Modeling steering within above-the-surface interaction layers. In CHI '07, 317--326. 2007. Google ScholarDigital Library
- J. Lee, A. Olwal, H. Ishii, and C. Boulanger. SpaceTop: integrating 2D and spatial 3D interactions in a see-through desktop environment. In CHI '13, 189--192. 2013. Google ScholarDigital Library
- N. Marquardt, R. Jota, S. Greenberg, and J. Jorge. The continuous interaction space: Interaction techniques unifying touch and gesture on and above a digital surface. In INTERACT '11, 461--476. 2011. Google ScholarDigital Library
- C. Mueller-Tomfelde, A. Wessels, and C. Schremmer. Tilted tabletops: In between horizontal and vertical workspaces. In Wksp on Horizontal Interactive Human Computer Systems, TABLETOP '08, 49--56. 2008.Google Scholar
- T. Mysliwiec. Fingermouse: A freehand computer pointing interface. In Int'l Conference on Automatic Face and Gesture Recognition, 372--377. 1994.Google Scholar
- M. Ortega and L. Nigay. Airmouse: Finger gesture for 2d and 3d interaction. In INTERACT '09, 214--227. 2009. Google ScholarDigital Library
- D. Pyryeskin, M. Hancock, and J. Hoey. Comparing Elicited Gestures to Designer-created Gestures for Selection Above a Multitouch Surface. In ITS '12, 1--10. 2012. Google ScholarDigital Library
- M. Spindler, M. Martsch, and R. Dachselt. Going beyond the surface: studying multi-layer interaction above the tabletop. In CHI '12, 1277--1286. 2012. Google ScholarDigital Library
- M. Spindler, S. Stellmach, and R. Dachselt. Paperlens: advanced magic lens interaction above the tabletop. In ITS '09, 69--76. 2009. Google ScholarDigital Library
- S. Subramanian, D. Aliakseyeu, and A. Lucero. Multi-layer interaction for digital tables. In UIST '06, 269--272. 2006. Google ScholarDigital Library
- D. Vogel and R. Balakrishnan. Distant freehand pointing and clicking on very large, high resolution displays. In UIST '05, 33--42. 2005. Google ScholarDigital Library
- A. D. Wilson. Robust computer vision-based detection of pinching for one and two-handed gesture input. In UIST '06, 255--258. 2006. Google ScholarDigital Library
- A. D. Wilson and H. Benko. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In UIST '10, 273--282. 2010. Google ScholarDigital Library
- J. O. Wobbrock, L. Findlater, D. Gergle, and J. J. Higgins. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In CHI '11, 143--146. 2011. Google ScholarDigital Library
- C. Yu, X. Tan, Y. Shi, and Y. Shi. Air finger: enabling multi-scale navigation by finger height above the surface. In UbiComp '11, 495--496. 2011. Google ScholarDigital Library
Index Terms
- Understanding finger input above desktop devices
Recommendations
iFAD Gestures: Understanding Users’ Gesture Input Performance with Index-Finger Augmentation Devices
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsWe examine gestures performed with a class of input devices with distinctive quality properties in the wearables landscape, which we call “index-Finger Augmentation Devices” (iFADs). We introduce a four-level taxonomy to characterize the diversity of ...
Touch+Finger: Extending Touch-based User Interface Capabilities with "Idle" Finger Gestures in the Air
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and TechnologyIn this paper, we present Touch+Finger, a new interaction technique that augments touch input with multi-finger gestures for rich and expressive interaction. The main idea is that while one finger is engaged in a touch event, a user can leverage the ...
FittsFarm: Comparing Children’s Drag-and-Drop Performance Using Finger and Stylus Input on Tablets
Human-Computer Interaction – INTERACT 2019AbstractWe used a two-dimensional Fitts’ law task to compare finger and stylus input with children when performing drag-and-drop tasks on a tablet. Twenty-eight children completed the study. Drag-and-drop performance was significantly better using a low-...
Comments