Majed Al Zayer
Eelke Folmer
ABSTRACT
Hand input offers a natural, efficient and immersive form of input for virtual reality (VR), but it has been difficult to implement on mobile VR platforms. Accurate hand-tracking requires a depth sensor and performing computer vision on a smartphone is computationally intensive, which may degrade the frame rate of a VR simulation and drain battery life. PAWdio is a novel 1 degree of freedom (DOF) hand input technique that uses acoustic sensing to track the relative position of an earbud from a headset that the user holds in their hand. PAWdio requires no instrumentation and its low computational overhead assures a high frame rate. A user study with 18 subjects evaluates PAWdio with button input that is commonly available on VR adapters. Results with a 3D target selection task found a similar accuracy and usability, a significantly slower performance, but higher immersion for PAWdio. We discuss limitations and game applications of PAWdio.
Supplemental Material
- Acoustic Ruler, iPhone App https://itunes.apple.com/us/app/acoustic-ruler-pro/id475081963?mt=8.Google Scholar
- Android hand tracking in OpenCV, https://www.youtube.com/watch?v=1pZbY8naj9Q.Google Scholar
- Stack overflow: Playing sound over speakers while playing music through headphones http://stackoverflow.com/questions/12017297/playing-sound-over-speakers-while-playing-music-through-headphones.Google Scholar
- Tarsos: Real-time audio processing framework in java, https://github.com/JorenSix/TarsosDSP.Google Scholar
- Works with google cardboard guidelines and best practices, http://static.googleusercontent.com/media/www.google.com/en//get/cardboard/downloads/wwgc_best_practices.pdf.Google Scholar
- Barbagallo, R. How to support gear and google cardboard in one unity3d project, http://ralphbarbagallo.com/2015/05/26/how-to-support-gear-vr-and-google-cardboard-in-one-unity3d-project/.Google Scholar
- Bowman, D. A., Kruijff, E., LaViola Jr, J. J., and Poupyrev, I. 3D user interfaces: theory and practice. Addison-Wesley, 2004. Google ScholarDigital Library
- Clemons, J., Zhu, H., Savarese, S., and Austin, T. Mevbench: A mobile computer vision benchmarking suite. In Workload Characterization (IISWC), 2011 IEEE International Symposium on, IEEE (2011), 91--102. Google ScholarDigital Library
- Gowases, T., Bednarik, R., and Tukiainen, M. Gaze vs. mouse in games: The effects on user experience. Proceedings of the 16th International Conference on Computers in Education (ICCE 2008) (2008), 773--777.Google Scholar
- Gupta, S., Morris, D., Patel, S., and Tan, D. Soundwave: using the doppler effect to sense gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2012), 1911--1914. Google ScholarDigital Library
- Harrison, C., Tan, D., and Morris, D. Skinput: appropriating the body as an input surface. In Proc. of the 28th international conference on Human factors in computing systems (CHI '10), ACM (2010), 453--462. Google ScholarDigital Library
- Kalgaonkar, K., and Raj, B. One-handed gesture recognition using ultrasonic doppler sonar. In Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, IEEE (2009), 1889--1892. Google ScholarDigital Library
- Kopper, R., Bowman, D. A., Silva, M. G., and McMahan, R. P. A human motor behavior model for distal pointing tasks. International journal of human-computer studies 68, 10 (2010), 603--615. Google ScholarDigital Library
- Laput, G., Brockmeyer, E., Hudson, S. E., and Harrison, C. Acoustruments: Passive, acoustically-driven, interactive controls for handheld devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15), ACM (2015), 2161--2170. Google ScholarDigital Library
- Laput, G., Chen, X. A., and Harrison, C. Sweepsense: Ad hoc configuration sensing using reflected swept-frequency ultrasonics. In Proc. of the 21st International Conference on Intelligent User Interfaces (IUI'16), ACM (2016), 332--335. Google ScholarDigital Library
- Lazik, P., and Rowe, A. Indoor pseudo-ranging of mobile devices using ultrasonic chirps. In Proc. of the 10th ACM Conference on Embedded Network Sensor Systems, ACM (2012), 99--112. Google ScholarDigital Library
- Meehan, M., Razzaque, S., Whitton, M. C., and Brooks Jr, F. P. Effect of latency on presence in stressful virtual environments. In Proc. IEEE Virtual Reality, IEEE (2003), 141--148. Google ScholarDigital Library
- Peng, C., Shen, G., Zhang, Y., Li, Y., and Tan, K. Beepbeep: a high accuracy acoustic ranging system using cots mobile devices. In Proc. of the 5th international conference on Embedded networked sensor systems, ACM (2007), 1--14. Google ScholarDigital Library
- Qifan, Y., Hao, T., Xuebing, Z., Yin, L., and Sanfeng, Z. Dolphin: Ultrasonic-based gesture recognition on smartphone platform. In IEEE International Conference on Computational Science and Engineering (CSE), IEEE (2014), 1461--1468. Google ScholarDigital Library
- Qiu, J., Chu, D., Meng, X., and Moscibroda, T. On the feasibility of real-time phone-to-phone 3d localization. In Proc. of the 9th ACM Conference on Embedded Networked Sensor Systems, ACM (2011), 190--203. Google ScholarDigital Library
- Raj, B., Kalgaonkar, K., Harrison, C., and Dietz, P. Ultrasonic doppler sensing in hci. IEEE Pervasive Computing, 2 (2012), 24--29. Google ScholarDigital Library
- Smus, B., and Riederer, C. Magnetic input for mobile virtual reality. In Proc. of the International Symposium on Wearable Computers, ACM (2015), 43--44. Google ScholarDigital Library
- Soukoreff, R. W., and MacKenzie, I. S. Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI. International journal of human-computer studies 61, 6 (2004), 751--789. Google ScholarDigital Library
- Stelmachowicz, P. G., Beauchaine, K. A., Kalberer, A., and Jesteadt, W. Normative thresholds in the 8-to 20-khz range as a function of age. The Journal of the Acoustical Society of America 86, 4 (1989), 1384--1391.Google ScholarCross Ref
- Teather, R. J., and Stuerzlinger, W. Pointing at 3d targets in a stereo head-tracked virtual environment. In IEEE Symposium on 3D User Interfaces, (2011), 87--94. Google ScholarDigital Library
- Tregillus, S., and Folmer, E. VR-step: Walking-in-place using inertial sensing for hands free navigation in mobile VR environments. In Proc. of the Conference on Human Factors in Computing Systems (CHI'15), (ACM 2016), 1250--1255. Google ScholarDigital Library
- Witmer, B. G., and Singer, M. J. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and virtual environments 7, 3 (1998), 225--240. Google ScholarDigital Library
- Yun, S., Chen, Y.-C., and Qiu, L. Turning a mobile device into a mouse in the air. In Proc. of the 13th Annual International Conference on Mobile Systems, Applications, and Services, ACM (2015), 15--29. Google ScholarDigital Library
Index Terms
- PAWdio: Hand Input for Mobile VR using Acoustic Sensing
Recommendations
VR-STEP: Walking-in-Place using Inertial Sensing for Hands Free Navigation in Mobile VR Environments
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing SystemsLow-cost smartphone adapters can bring virtual reality to the masses, but input is typically limited to using head tracking, which makes it difficult to perform complex tasks like navigation. Walking-in-place (WIP) offers a natural and immersive form of ...
Do virtual reality head-mounted displays make a difference? A comparison of presence and self-efficacy between head-mounted displays and desktop computer-facilitated virtual environments
AbstractVirtual reality (VR) has made it possible for users to access novel digital experiences. An interesting question that arises in the context of VR is whether it appears or feels different to users when different virtual environments are used. This ...
Towards integration of user-centered designed tutorials for better virtual reality immersion
ICIGP '19: Proceedings of the 2nd International Conference on Image and Graphics ProcessingVirtual reality (VR) has been contributing to education, health sciences and entertainment of late. The technology has made itself flexible enough to cater to users of different ages. Additionally head-mounted displays (HMDs) has led to the increase of ...
Comments