Zhen Li
Joannes Chan
Hrvoje Benko
Daniel Wigdor
Michael Glueck
ABSTRACT
In virtual reality (VR) environments, asymmetric bimanual interaction techniques can increase users’ input bandwidth by complementing their perceptual and motor systems (e.g., using the dominant hand to select 3D UI controls anchored around the non-dominant arm). However, it is unclear how to optimize the layout of such 3D UI controls for near-body and mid-air interactions. We evaluate the performance and limitations of non-dominant arm-anchored 3D UIs in VR environments through a bimanual pointing study. Results demonstrated that targets appearing closer to the skin, located around the wrist, or placed on the medial side of the forearm could be selected more quickly than targets farther away from the skin, located around the elbow, or on the lateral side of the forearm. Based on these results, we developed Armstrong guidelines, demonstrated through a Unity plugin to enable designers to create performance-optimized arm-anchored 3D UI layouts.
Supplemental Material
Available for Download
- Apple Inc.2020. App Icon - Visual - WatchOS. https://developer.apple.com/design/human-interface-guidelines/watchos/visual/app-icon/Google Scholar
- Apple Inc.2020. Change Cursor preferences for accessibility on Mac. https://support.apple.com/en-ca/guide/mac-help/mchl5bb12e1e/macGoogle Scholar
- K. S. Arun, T. S. Huang, and S. D. Blostein. 1987. Least-Squares Fitting of Two 3-D Point Sets. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-9, 5(1987), 698–700.Google ScholarDigital Library
- Takumi Azai, Shuhei Ogawa, Mai Otsuki, Fumihisa Shibata, and Asako Kimura. 2017. Selection and Manipulation Methods for a Menu Widget on the Human Forearm. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI ’17 EA). 357–360. https://doi.org/10.1145/3027063.3052959Google ScholarDigital Library
- Takumi Azai, Mai Otsuki, Fumihisa Shibata, and Asako Kimura. 2018. Open Palm Menu: A Virtual Menu Placed in Front of the Palm. In Proceedings of the 9th Augmented Human International Conference (AH ’18). 1–5. https://doi.org/10.1145/3174910.3174929Google ScholarDigital Library
- Joanna Bergstrom-Lehtovirta, Sebastian Boring, and Kasper Hornbæk. 2017. Placing and Recalling Virtual Items on the Skin. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’17). 1497–1507. https://doi.org/10.1145/3025453.3026030Google ScholarDigital Library
- Joanna Bergstrom-Lehtovirta, Kasper Hornbæk, and Sebastian Boring. 2018. It’s a Wrap : Mapping On-Skin Input to Off-Skin Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’18). 1–11. https://doi.org/10.1145/3173574.3174138Google ScholarDigital Library
- Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. 1993. Toolglass and magic lenses: the see-through interface. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 73–80. https://doi.org/10.1145/166117.166126Google ScholarDigital Library
- Renaud Blanch and Michael Ortega. 2011. Benchmarking pointing techniques with distractors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11). 1629–1638. https://doi.org/10.1145/1978942.1979180Google ScholarDigital Library
- D.A. Bowman and C.A. Wingrave. 2001. Design and evaluation of menu systems for immersive virtual environments. In Proceedings IEEE Virtual Reality 2001. 149–156. https://doi.org/10.1109/vr.2001.913781Google ScholarCross Ref
- Olivier Chapuis, Renaud Blanch, and Michel Beaudouin-lafon. 2007. Fitts ’ Law in the Wild : A Field Study of Aimed Movements. Technical Report. https://hal.archives-ouvertes.fr/hal-00612026Google Scholar
- Justine Cléry, Olivier Guipponi, Claire Wardak, and Suliann Ben Hamed. 2015. Neuronal bases of peripersonal and extrapersonal spaces, their plasticity and their dynamics: Knowns and unknowns. Neuropsychologia 70(2015), 313–326. https://doi.org/10.1016/j.neuropsychologia.2014.10.022Google ScholarCross Ref
- Raimund Dachselt and Anett Hübner. 2006. A Survey and Taxonomy of 3D Menu Techniques. In Proceedings of the 12th Eurographics Symposium on Virtual Environments (EGVE ’06). 89–99.Google ScholarDigital Library
- Niloofar Dezfuli, Mohammadreza Khalilbeigi, Jochen Huber, Florian Benjamin Müller, and Max Mühlhäuser. 2012. PalmRC: Imaginary Plam-based Remote Control for Eyes-free Television Interaction. In Proceedings of the 10th European Conference on Interactive TV and Video (EuroITV ’12). 27–34. https://doi.org/10.1080/0144929X.2013.810781Google ScholarDigital Library
- Sarah A. Douglas, Arthur E. Kirkpatrick, and I. Scott MacKenzie. 1999. Testing pointing device performance and user assessment with the IS0 9241, Part 9 standard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’99). 215–222. https://doi.org/10.1145/302979.303042Google ScholarDigital Library
- Paul M. Fitts. 1954. The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement. Journal of Experimental Psychology 47, 6 (1954), 381–391. http://www2.psychology.uiowa.edu/faculty/mordkoff/InfoProc/pdfs/Fitts 1954.pdfGoogle ScholarCross Ref
- Google LLC. 2016. Tilt Brush by Google. https://www.tiltbrush.com/Google Scholar
- Google LLC. 2017. Blocks - Create 3D Models in VR. https://arvr.google.com/blocks/Google Scholar
- Michael S.A. Graziano. 1999. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proceedings of the National Academy of Sciences of the United States of America 96, 18 (1999), 10418–10421. https://doi.org/10.1073/pnas.96.18.10418Google ScholarCross Ref
- Yves Guiard. 1987. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior 19, 4 (1987), 486–517. https://doi.org/10.1080/00222895.1987.10735426Google ScholarCross Ref
- Sean Gustafson, Bernhard Rabe, and Patrick Baudisch. 2013. Understanding palm-based imaginary interfaces: the role of visual and tactile cues when browsing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). 889–898. https://doi.org/10.1145/2470654.2466114Google ScholarDigital Library
- Chris Harrison, Hrvoje Benko, and Andrew D. Wilson. 2011. OmniTouch: Wearable Multitouch Interaction Everywhere. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST ’11). 441–450. https://doi.org/10.1145/2047196.2047255Google ScholarDigital Library
- Chris Harrison, Shilpa Ramamurthy, and Scott E. Hudson. 2012. On-Body Interaction: Armed and Dangerous. In In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI ’12). 69–76. https://doi.org/10.1145/2148131.2148148Google ScholarDigital Library
- Chris Harrison, Desney Tan, and Dan Morris. 2010. Skinput: Appropriating the Body as an Input Surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). 453–462. https://doi.org/10.1145/1753326.1753394Google ScholarDigital Library
- Julian Hettig, Sandy Engelhardt, Christian Hansen, and Gabriel Mistelbauer. 2018. AR in VR: assessing surgical augmented reality visualizations in a steerable virtual reality environment. International Journal of Computer Assisted Radiology and Surgery 13, 11(2018), 1717–1725. https://doi.org/10.1007/s11548-018-1825-4Google ScholarCross Ref
- Ken Hinckley, Randy Pausch, and Dennis Proffitt. 1997. Attention and Visual Feedback: The Bimanual Frame of Reference. In Symposium on Interactive 3D Graphics. 121–126. https://doi.org/10.1145/253284.253318Google ScholarDigital Library
- Leap Motion Inc.2020. Documentation – Leap Motion Developer. https://developer.leapmotion.com/documentationGoogle Scholar
- Irina Lediaeva and Joseph J. LaViola. 2020. Evaluation of Body-Referenced Graphical Menus in Virtual Environments. In Proceedings of the Graphics Interface Conference 2020 (GI ’20). 1–9.Google Scholar
- Shu Yang Lin, Chao Huai Su, Kai Yin Cheng, Rong Hao Liang, Tzu Hao Kuo, and Bing Yu Chen. 2011. PUB - Point upon body: Exploring eyes-free interaction and methods on an arm. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST ’11). 481–487. https://doi.org/10.1145/2047196.2047259Google ScholarDigital Library
- I. Scott MacKenzie. 1992. Fitts’ Law as a Research and Design Tool in Human-Computer Interaction. Human–Computer Interaction 7, 1 (1992), 91–139. https://doi.org/10.1207/s15327051hci0701_3Google ScholarDigital Library
- I. Scott MacKenzie, Abigail Sellen, and William a S Buxton. 1991. A comparison of input devices in element pointing and dragging tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’91). 161–166. https://doi.org/10.1145/108844.108868Google ScholarDigital Library
- P. F. MacNeilage, M. G. Studdert-Kennedy, and B. Lindblom. 1984. Functional precursors to language and its lateralization. American Journal of Physiology - Regulatory Integrative and Comparative Physiology 15, 6 (1984), 912–914. https://doi.org/10.1152/ajpregu.1984.246.6.r912Google ScholarCross Ref
- Tamar R. Makin, Nicholas P. Holmes, and H. Henrik Ehrsson. 2008. On the other hand: Dummy hands and peripersonal space. Behavioural Brain Research 191, 1 (2008), 1–10. https://doi.org/10.1016/j.bbr.2008.02.041Google ScholarCross Ref
- Tamar R. Makin, Nicholas P. Holmes, and Ehud Zohary. 2007. Is that near my hand? Multisensory representation of peripersonal space in human intraparietal sulcus. Journal of Neuroscience 27, 4 (2007), 731–740. https://doi.org/10.1523/JNEUROSCI.3653-06.2007Google ScholarCross Ref
- D. I. McCloskey. 1978. Kinesthetic Sensibility. Physiological Reviews 58, 4 (1978), 763–811.Google ScholarCross Ref
- Tim Menzner, Travis Gesslein, Alexander Otte, and Jens Grubert. 2020. Above surface interaction for multiscale navigation in mobile virtual reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces. 372–381. https://doi.org/10.1109/vr46266.2020.00057Google ScholarCross Ref
- Microsoft Corporation. 2019. Hand Menu – Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/hand-menuGoogle Scholar
- Mark R Mine, Frederick P Brooks Jr, The Problem, and Frederick P Brooks Jr.1997. Moving Objects in Space : Exploiting Proprioception In Virtual-Environment Interaction. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques (SIGGRAPH ’97). 19–26. https://doi.org/10.1145/258734.258747Google ScholarDigital Library
- Florian Müller, Niloofar Dezfuli, Max Mühlhäuser, Martin Schmitz, and Mohammadreza Khalilbeigi. 2015. Palm-based Interaction with Head-mounted Displays. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI ’15). 963–965.Google ScholarDigital Library
- Atsuo Murata and Hirokazu Iwase. 2001. Extending Fitts’ law to a three-dimensional pointing task. Human movement science 20, 6 (2001), 791–805.Google Scholar
- NatureManufacture. 2017. VR Hands and FP Arms Pack. https://assetstore.unity.com/packages/3d/characters/humanoids/vr-hands-and-fp-arms-pack-77815#releasesGoogle Scholar
- Oculus Inc.2016. Oculus Rift remote. https://support.oculus.com/950065025058681/Google Scholar
- Masa Ogata, Yuta Sugiura, and Yasutoshi Makino. 2013. SenSkin: adapting skin as a soft interface. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST ’13). 539–543. https://doi.org/10.1145/2501988.2502039Google ScholarDigital Library
- Ken Pfeuffer, Benedikt Mayer, Diako Mardanbegi, and Hans Gellersen. 2017. Gaze + Pinch interaction in virtual reality. In Proceedings of the 2017 Symposium on Spatial User Interaction (SUI ’17). 99–108. https://doi.org/10.1145/3131277.3132180Google ScholarDigital Library
- Alexandros Pino, Evangelos Tzemis, Nikolaos Ioannou, and Georgios Kouroupetroglou. 2013. Using kinect for 2D and 3D pointing tasks: performance evaluation. In International Conference on Human-Computer Interaction. Springer, 358–367.Google ScholarDigital Library
- Catherine L. Reed, Jefferson D. Grubb, and Cleophus Steele. 2006. Hands up: Attentional priorization of space near the hand. Journal of Experimental Psychology: Human Perception and Performance 32, 1(2006), 166–177. https://doi.org/10.1037/0096-1523.32.1.166Google ScholarCross Ref
- Giacomo Rizzolatti, Luciano Fadiga, Leonardo Fogassi, and Vittorio Gallese. 1997. The Space Around Us. Science 277, 5323 (1997), 190–191. https://doi.org/10.1126/science.277.5323.190Google ScholarCross Ref
- Giacomo Rizzolatti, Cristiana Scandolara, Massimo Matelli, and Maurizio Gentilucci. 1981. Afferent properties of periarcuate neurons in macaque monkeys. II. Visual responses. Behavioural Brain Research 2, 2 (1981), 147–163. https://doi.org/10.1016/0166-4328(81)90053-XGoogle ScholarCross Ref
- George Robertson, Maarten van Dantzich, Daniel Robbins, Mary Czerwinski, Ken Hinckley, Kirsten Risden, David Thiel, and Vadim Gorokhovsky. 2000. The Task Gallery: A 3D Window Manager. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’00). 494–501. https://doi.org/10.1145/332040.332482Google ScholarDigital Library
- RootMotion. 2020. Final IK. https://assetstore.unity.com/packages/tools/animation/final-ik-14290Google Scholar
- Stefan Schneegass and Alexandra Voit. 2016. GestureSleeve: Using Touch Sensitive Fabrics for Gestural Input on the Forearm for Controlling Smartwatches. In Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC ’16). 108–115. https://doi.org/10.1145/2971763.2971797Google ScholarDigital Library
- Raymond Scupin. 1997. The KJ method: A technique for analyzing data derived from Japanese ethnology. Human Organization 56, 2 (1997), 233–237. https://doi.org/10.17730/humo.56.2.x335923511444655Google ScholarCross Ref
- C. S. Sherrington. 1907. On the Proprio-ceptive System, especially in its Reflex Aspect. Brain 29, 4 (1907), 467–482.Google ScholarCross Ref
- Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual reality on a WIM: interactive worlds in miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’95). 265–272.Google ScholarDigital Library
- Sriram Subramanian, Dzimitry Aliakseyeu, and Andrés Lucero. 2006. Multi-layer interaction for digital tables. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST ’06). 269–272. https://doi.org/10.1145/1166253.1166295Google ScholarDigital Library
- J. L. Taylor. 2009. Proprioception. Academic Press. 1143–1149 pages. https://doi.org/10.1016/B978-008045046-9.01907-0.Google ScholarCross Ref
- Robert J Teather and Wolfgang Stuerzlinger. 2013. Pointing at 3d target projections with one-eyed and stereo cursors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 159–168.Google ScholarDigital Library
- Kashyap Todi, Daryl Weir, and Antti Oulasvirta. 2016. Sketchplore: Sketch and explore with a layout optimiser. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS ’16). 543–555. https://doi.org/10.1145/2901790.2901817Google ScholarDigital Library
- Velko Vechev, Alexandru Dancu, Simon T. Perrault, Quentin Roy, Morten Fjeld, and Shengdong Zhao. 2018. Movespace: on-body athletic interaction for running and cycling. In Proceedings of the 2018 International Conference on Advanced Visual Interfaces (AVI ’18). 1–9. https://doi.org/10.1145/3206505.3206527Google ScholarDigital Library
- Julie Wagner, Mathieu Nancel, Sean G. Gustafson, Stephane Huot, and Wendy E. Mackay. 2013. Body-centric design space for multi-surface interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). 1299–1308. https://doi.org/10.1145/2470654.2466170Google ScholarDigital Library
- Cheng-Yao Wang, Wei-Chen Chu, Po-Tsung Chiu, Min-Chieh Hsiu, Yih-Harn Chiang, and Mike Y. Chen. 2015. PalmType: Using Palms as Keyboards for Smart Glasses. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’15). 153–160. https://doi.org/10.1145/2785830.2785886Google ScholarDigital Library
- Cheng-Yao Wang, Min-Chieh Hsiu, Po-Tsung Chiu, Chiao-Hui Chang, Liwei Chan, Bing-Yu Chen, and Mike Y. Chen. 2015. Palm gesture: Using palms as gesture interfaces for eyes-free input. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’15). 217–226. https://doi.org/10.1145/2785830.2785885Google ScholarDigital Library
- Martin Weigel, Mehta Vikram, and Jürgen Steimle. 2014. More Than Touch: Understanding How People Use Skin as an Input Surface for Mobile Computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). 179–188. https://doi.org/10.1145/2556288.2557239Google ScholarDigital Library
- Xuhai Xu, Alexandru Dancu, Pattie Maes, and Suranga Nanayakkara. 2018. Hand range interface: information always at hand with a body-centric mid-air input surface. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI ’18). 1–12. https://doi.org/10.1145/3229434.3229449Google ScholarDigital Library
- Yukang Yan, Chun Yu, Xiaojuan Ma, Shuai Huang, Hasan Iqbal, and Yuanchun Shi. 2018. Eyes-Free Target Acquisition in Interaction Space around the Body for Virtual Reality. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’18). 1–13. https://doi.org/10.1145/3173574.3173616Google ScholarDigital Library
- Shumin Zhai, Michael Hunter, and Barton Smith. 2002. Performance Optimization of Virtual Keyboards. Human-Computer Interaction 17, 2 (2002), 229–269. https://doi.org/10.1207/S15327051HCI172&3_4Google ScholarCross Ref
- Yang Zhang, Wolf Kienzle, Yanjun Ma, Shiu S Ng, Hrvoje Benko, and Chris Harrison. 2019. ActiTouch : Robust Touch Detection for On-Skin AR / VR Interfaces. In Proceedings of the ACM Symposium on User Interface Software & Technology (UIST ’19). 1151–1159.Google ScholarDigital Library
Index Terms
- Armstrong: An Empirical Examination of Pointing at Non-Dominant Arm-Anchored UIs in Virtual Reality
Index terms have been assigned to the content through auto-classification.
Recommendations
An evaluation of asymmetric interfaces for bimanual virtual assembly with haptics
Immersive computing technology provides a human---computer interface to support natural human interaction with digital data and models. One application for this technology is product assembly methods planning and validation. This paper presents the ...
Nomadic Virtual Reality: Exploring New Interaction Concepts for Mobile Virtual Reality Head-Mounted Displays
UIST '16 Adjunct: Adjunct Proceedings of the 29th Annual ACM Symposium on User Interface Software and TechnologyTechnical progress and miniaturization enables virtual reality (VR) head-mounted displays (HMDs) now to be solely operated using a smartphone as a display, processing unit and sensor unit. These mobile VR HMDs (e.g. Samsung GearVR) allow for a whole new ...
Towards usable VR: an empirical study of user interfaces for immersive virtual environments
CHI '99: Proceedings of the SIGCHI conference on Human Factors in Computing SystemsThis paper reports empirical results from a study into the use of 2D widgets in 3D immersive virtual environments. Several researchers have proposed the use of 2D interaction techniques in 3D environments, however little empirical work has been done to test ...
Comments