Skip to main content
SpringerLink
Log in

Embodied Interaction on Constrained Interfaces for Augmented Reality

  • Chapter
  • First Online:
Springer Handbook of Augmented Reality

Part of the book series: Springer Handbooks ((SHB))

Abstract

Wearable computers have seen a recent resurgence in interest and popularity in which augmented reality (AR) smartglasses are poised to influence the way we complete our work and daily tasks. Nowadays, industrial applications of these smartglasses are focused on interior designs, remote collaborations, as well as e-commerce. Under five key constraints on AR smartglasses such as miniature touch interface, small-screen real estate, user mobility, limited computational resource, and short battery life, existing user interaction paradigms designed for desktop computers and smartphones are obsolete and incompatible with the scenarios of AR smartglasses. The cumbersome and difficult interaction with the AR smartglasses becomes a hurdle to their wider industrial applications. Thus, an unmet demand for designing interaction techniques on AR smartglasses is undoubtedly critical.

In this chapter, we present three interaction techniques, namely, TiPoint, HIBEY, and TOFI, in order to enhance object manipulation and text entry in the constrained environment of AR smartglasses. These techniques are devised in a way that leverage on advantageous features of human body and experiences such as the dexterity of fingertip, lexicographical order ingrained in our memory, proprioception, as well as opposable thumbs. We thoroughly address the key constraints on AR smartglasses and explore different modalities with various hardware and peripherals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 309.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 399.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lee, L., Hui, P.: Interaction methods for smart glasses: a survey. IEEE Access 6, 28712–28732 (2018)

    Article  Google Scholar 

  2. Singh, G., Delamare, W., Irani, P.: D-swime: a design space for smartwatch interaction techniques supporting mobility and encumbrance. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pp. 634:1–634:13, Montreal QC, Canada. ACM, New York (2018)

    Google Scholar 

  3. Blattgerste, J., Strenge, B., Renner, P., Pfeiffer, T., Essig, K.: Comparing conventional and augmented reality instructions for manual assembly tasks. In: Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA ’17, pp 75–82, Island of Rhodes, Greece. ACM, New York (2017)

    Google Scholar 

  4. Hsieh, Y.T., Jylhä, A., Orso, V., Gamberini, L., Jacucci, G.: Designing a willing-to-use-in-public hand gestural interaction technique for smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, pp. 4203–4215, San Jose, California, USA. ACM, New York (2016)

    Google Scholar 

  5. Bispham, M.: Security vulnerabilities in speech recognition systems. Technical report, 2016

    Google Scholar 

  6. Moggridge, B.: Designing Interactions. The MIT Press, Cambridge (2006)

    Google Scholar 

  7. Whitmire, E., Jain, M., Jain, D., Nelson, G., Karkar, R., Patel, S., Goel, M.: Digitouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-Mounted Displays, vol. 1, pp. 113:1–113:21, Maui, Hawaii, USA. ACM, New York (2017)

    Google Scholar 

  8. O’Sullivan, D., Igoe, T.: Physical Computing: Sensing and Controlling the Physical World with Computers. Course Technology Press, Boston (2004)

    Google Scholar 

  9. Leigh, S.-W., Sareen, H., Kao, H.-L.-C., Liu, X., Maes, P.: Body-borne computers as extensions of self. Computers 6(1) (2017)

    Google Scholar 

  10. Friston, K.J., Daunizeau, J., Kilner, J., Kiebel, S.J.: Action and behavior: a free-energy formulation. Biol. Cybern. 102(3), 227–260 (2010)

    Article  Google Scholar 

  11. Jacob, R.J., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T., Zigelbaum, J.: Reality-based interaction: a framework for post-wimp interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, pp. 201–210, Florence, Italy. ACM, New York (2008)

    Google Scholar 

  12. Garg, A.B.: Embodied cognition, human computer interaction, and application areas. In: Kim, T.-H., Mohammed, S., Ramos, C., Abawajy, J., Kang, B.-H., Ślezak, D. (eds.) Computer Applications for Web, Human Computer Interaction, Signal and Image Processing, and Pattern Recognition, pp. 369–374, Berlin, Heidelberg. Springer, Berlin (2012)

    Google Scholar 

  13. Dourish, P.: “Being-in-the-World”: Embodied Interaction. MITP (2004)

    Google Scholar 

  14. Lee, L.H., Braud, T., Bijarbooneh, F.H., Hui, P.:. Tipoint: detecting fingertip for mid-air interaction on computational resource constrained smartglasses. In: Proceedings of the 23rd International Symposium on Wearable Computers, ISWC ’19, pp. 118–122, New York, NY, USA. ACM, New York (2019)

    Google Scholar 

  15. Yau, Y.P., Braud, T., Lee, L.-H., Lam, K.Y., Hui, P.: Hibey: Hide the keyboard in augmented reality. In: Proceedings of the IEEE International Conference on Pervasive Computing and Communications, PERCOM ’19, Kyoto, Japan. IEEE (2019)

    Google Scholar 

  16. Lee, L.H., Lam, K.Y., Li, T., Braud, T., Su, X., Hui, P.: Quadmetric optimized thumb-to-finger interaction for force assisted one-handed text entry on mobile headsets. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(3), 94:1–94:27 (2019)

    Google Scholar 

  17. Tung, Y.-C., Hsu, C.-Y., Wang, H.-Y., Chyou, S., Lin, J.-W., Wu, P.-J., Valstar, A., Chen, M.Y.: User-defined game input for smart glasses in public space. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 3327–3336, Seoul, Republic of Korea. ACM, New York (2015)

    Google Scholar 

  18. Grossman, T., Chen, X.A., Fitzmaurice, G.: Typing on glasses: adapting text entry to smart eyewear. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15, pp. 144–152, Copenhagen, Denmark. ACM, New York (2015)

    Google Scholar 

  19. Kienzle, W., Hinckley, K.: Lightring: always-available 2d input on any surface. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14, pp. 157–160, Honolulu, Hawaii, USA. ACM, New York (2014)

    Google Scholar 

  20. Yang, X.-D., Grossman, T., Wigdor, D., Fitzmaurice, G.: Magic finger: always-available input through finger instrumentation. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST ’12, pp. 147–156, Cambridge, Massachusetts, USA. ACM, New York (2012)

    Google Scholar 

  21. Ogata, M., Sugiura, Y., Osawa, H., Imai, M.: iring: intelligent ring using infrared reflection. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, UIST ’12, pp. 131–136, Cambridge, Massachusetts, USA. ACM, New York (2012)

    Google Scholar 

  22. Ashbrook, D., Baudisch, P., White, S.: Nenya: Subtle and eyes-free mobile input with a magnetically-tracked finger ring. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, pp. 2043–2046, Vancouver, BC, Canada. ACM, New York (2011)

    Google Scholar 

  23. Rekimoto, J.: Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In: Proceedings of the 5th IEEE International Symposium on Wearable Computers, ISWC ’01, pp. 21–, Zurich, Switzerland. IEEE Computer Society (2001)

    Google Scholar 

  24. Ham, J., Hong, J., Jang, Y., Ko, S.H., Woo, W.: Smart Wristband: Touch-and-Motion–Tracking Wearable 3D Input Device for Smart Glasses, pp. 109–118. Springer, Berlin (2014)

    Google Scholar 

  25. Schneegass, S., Voit, A.: Gesturesleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches. In: Proceedings of the 2016 ACM International Symposium on Wearable Computers, ISWC ’16, pp. 108–115, Heidelberg, Germany. ACM, New York (2016)

    Google Scholar 

  26. Dobbelstein, D., Hock, P., Rukzio, E.: Belt: an unobtrusive touch input device for head-worn displays. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 2135–2138, Seoul, Republic of Korea. ACM, New York (2015)

    Google Scholar 

  27. Gustafson, S.G., Rabe, B., Baudisch, P.M.: Understanding palm-based imaginary interfaces: the role of visual and tactile cues when browsing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, pp. 889–898, Paris, France. ACM, New York (2013)

    Google Scholar 

  28. Harrison, C., Benko, H., Wilson, A.D.: Omnitouch: wearable multitouch interaction everywhere. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pp. 441–450, Santa Barbara, California, USA. ACM, New York (2011)

    Google Scholar 

  29. Harrison, C., Tan, D., Morris, D.: Skinput: appropriating the skin as an interactive canvas. Commun. ACM 54(8), 111–118 (2011)

    Article  Google Scholar 

  30. Weigel, M., Lu, T., Bailly, G., Oulasvirta, A., Majidi, C., Steimle, J.: iskin: flexible, stretchable and visually customizable on-body touch sensors for mobile computing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 2991–3000, Seoul, Republic of Korea. ACM, New York (2015)

    Google Scholar 

  31. Wang, C.-Y., Chu, W.-C., Chiu, P.-T., Hsiu, M.-C., Chiang, Y.-H., Chen, M.Y.: Palmtype: using palms as keyboards for smart glasses. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15, pp. 153–160, Copenhagen, Denmark. ACM, New York (2015)

    Google Scholar 

  32. Wang, C.-Y., Hsiu, M.-C., Chiu, P.-T., Chang, C.-H., Chan, L., Chen, B.-Y., Chen, M.Y.: Palmgesture: using palms as gesture interfaces for eyes-free input. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15, pp. 217–226, Copenhagen, Denmark. ACM, New York (2015)

    Google Scholar 

  33. Azai, T., Ogawa, S., Otsuki, M., Shibata, F., Kimura, A.: Selection and manipulation methods for a menu widget on the human forearm. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA ’17, pp. 357–360, Denver, Colorado, USA. ACM, New York (2017)

    Google Scholar 

  34. Ogata, M., Sugiura, Y., Makino, Y., Inami, M., Imai, M.: Senskin: adapting skin as a soft interface. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST ’13, pp. 539–544, St. Andrews, Scotland, United Kingdom. ACM, New York (2013)

    Google Scholar 

  35. Lin, S.-Y., Su, C.-H., Cheng, K.-Y., Liang, R.-H., Kuo, T.-H., Chen, B.-Y.: Pub-point upon body: Exploring eyes-free interaction and methods on an arm. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pp. 481–488, Santa Barbara, California, USA. ACM, New York (2011)

    Google Scholar 

  36. Huang, D.-Y., Chan, L., Yang, S., Wang, F., Liang, R.-H., Yang, D.-N., Hung, Y.-P., Chen, B.-Y.: Digitspace: designing thumb-to-fingers touch interfaces for one-handed and eyes-free interactions. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, pp. 1526–1537, San Jose, California, USA. ACM, New York (2016)

    Google Scholar 

  37. Yoon, S.H., Huo, K., Nguyen, V.P., Ramani, K.: Timmi: finger-worn textile input device with multimodal sensing in mobile interaction. In: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, TEI ’15, pp. 269–272, Stanford, California, USA. ACM, New York (2015)

    Google Scholar 

  38. Serrano, M., Ens, B.M., Irani, P.P.: Exploring the use of hand-to-face input for interacting with head-worn displays. In: Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems, CHI ’14, pp. 3181–3190, Toronto, Ontario, Canada. ACM, New York (2014)

    Google Scholar 

  39. Lissermann, R., Huber, J., Hadjakos, A., Mühlhäuser, M.: Earput: augmenting behind-the-ear devices for ear-based interaction. In: CHI ’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’13, pp. 1323–1328, Paris, France. ACM, New York (2013)

    Google Scholar 

  40. Jones, E., Alexander, J., Andreou, A., Irani, P., Subramanian, S.: Gestext: accelerometer-based gestural text-entry systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, pp. 2173–2182, Atlanta, Georgia, USA. ACM, New York (2010)

    Google Scholar 

  41. Shimizu, J., Chernyshov, G.: Eye movement interactions in google cardboard using a low cost EOG setup. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, UbiComp ’16, pp. 1773–1776, Heidelberg, Germany. ACM, New York (2016)

    Google Scholar 

  42. Schuchert, T., Voth, S., Baumgarten, J.: Sensing visual attention using an interactive bidirectional HMD. In: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction, Gaze-In ’12, pp. 16:1–16:3, Santa Monica, California. ACM, New York (2012)

    Google Scholar 

  43. Saponas, T.S., Kelly, D., Parviz, B.A., Tan, D.S.: Optically sensing tongue gestures for computer input. In: Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology, UIST ’09, pp. 177–180, Victoria, BC, Canada. ACM, New York (2009)

    Google Scholar 

  44. van Krevelen, D.W.F., Poelman, R.: A survey of augmented reality technologies, applications and limitations. Int. J. Virtual Reality 9(2), 1–20 (2010)

    Article  Google Scholar 

  45. Aigner, R., et al.: Understanding mid-air hand gestures: a study of human preferences in usage of gesture types for hci. Technical report, November 2012

    Google Scholar 

  46. Haque, F., Nancel, M., Vogel, D.: Myopoint: pointing and clicking using forearm mounted electromyography and inertial motion sensors. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 3653–3656, Seoul, Republic of Korea. ACM, New York (2015)

    Google Scholar 

  47. Hsieh, Y.-T., Jylhä, A., Jacucci, G.: Pointing and Selecting with Tactile Glove in 3D Environment, pp. 133–137. Springer, Berlin (2014)

    Google Scholar 

  48. Rosenberg, R., Slater, M.: The chording glove: a glove-based text input device. Trans. Sys. Man Cyber Part C 29(2), 186–191 (1999)

    Article  Google Scholar 

  49. Markussen, A., Jakobsen, M.R. and Hornbæk, K.: Vulture: a mid-air word-gesture keyboard. In: Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems, CHI ’14, pp. 1073–1082, Toronto, Ontario, Canada. ACM, New York (2014)

    Google Scholar 

  50. Chan, L., Chen, Y.-L., Hsieh, C.-H., Liang, R.-H., Chen, B.Y.: Cyclopsring: enabling whole-hand and context-aware interactions through a fisheye ring. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST ’15, pp. 549–556, Charlotte, NC, USA. ACM, New York (2015)

    Google Scholar 

  51. Huang, Z., Li, W., Hui, P.: Ubii: towards seamless interaction between digital and physical worlds. In: Proceedings of the 23rd ACM International Conference on Multimedia, MM ’15, pp. 341–350, Brisbane, Australia. ACM, New York (2015)

    Google Scholar 

  52. Lee, T., Hollerer, T.: Handy ar: markerless inspection of augmented reality objects using fingertip tracking. In: 2007 11th IEEE International Symposium on Wearable Computers, pp. 83–90, Boston, MA, USA (2007)

    Google Scholar 

  53. Bailly, G., Müller, J., Rohs, M., Wigdor, D. and Kratz, S.: Shoesense: a new perspective on gestural interaction and wearable applications. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pp. 1239–1248, Austin, Texas, USA. ACM, New York (2012)

    Google Scholar 

  54. Hollerer, T., et al.: Vision-based interfaces for mobility. In: The First Annual International Conference on Mobile and Ubiquitous Systems, vol. 00, pp. 86–94, Los Alamitos, CA, USA. IEEE Computer Society (2004)

    Google Scholar 

  55. Guimbretière, F., Nguyen, C.: Bimanual marking menu for near surface interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pp. 825–828, Austin, Texas, USA. ACM, New York (2012)

    Google Scholar 

  56. Ha, T., Feiner, S., Woo, W.: Wearhand: Head-worn, rgb-d camera-based, bare-hand user interface with visually enhanced depth perception. In: 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 219–228 (2014)

    Google Scholar 

  57. Heun, V., Kasahara, S., Maes, P.: Smarter objects: Using ar technology to program physical objects and their interactions. In: CHI ’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’13, pp. 2939–2942, Paris, France. ACM, New York (2013)

    Google Scholar 

  58. Pino, A., Tzemis, E., Ioannou, N., Kouroupetroglou, G.: Using kinect for 2d and 3d pointing tasks: performance evaluation. In: International Conference on Human-Computer Interaction, pp. 358–367, Las Vegas, NV, USA. Springer, Berlin (2013)

    Google Scholar 

  59. Ren, G.,O’Neill, E.: Freehand gestural text entry for interactive tv. In: Proceedings of the 11th European Conference on Interactive TV and Video, EuroITV ’13, pp. 121–130, Como, Italy. ACM, New York (2013)

    Google Scholar 

  60. Hamette, P., Lukowicz, P., Troster, G., Svoboda, T.: Fingermouse: A wearable hand tracking system. In: Adjunct Proceedings of the 4th International Conference on Ubiquitous Computing, Tokyo, Japan (2002)

    Google Scholar 

  61. Huang, Y., Liu, X., Zhang, X., Jin, L.: A pointing gesture based egocentric interaction system: dataset, approach and application. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 370–377, Las Vegas, USA. IEEE Computer Society (2016)

    Google Scholar 

  62. Simion, G., David, C., Gui, V., Caleanu, C.D.: Fingertip-based real time tracking and gesture recognition for natural user interfaces. Acta Polytech. Hung. 13, 189–204 (2016)

    Google Scholar 

  63. Howell Istance, Richard Bates, Aulikki Hyrskykari, and Stephen Vickers. Snap clutch, a moded approach to solving the midas touch problem. In: Proceedings of the 2008 Symposium on Eye Tracking Research & #38; Applications, vol. 38, pp. 221–228, Savannah, Georgia, USA (2008)

    Google Scholar 

  64. Ren, G., O’Neill, E.: 3d selection with freehand gesture. Comput. Graph. 37(3), 101–120 (2013)

    Article  Google Scholar 

  65. Heo, G., Lee, D.W., Shin, H.C., Jeong, H.-T., Yoo, T.-W.: Hand segmentation and fingertip detection for interfacing of stereo vision-based smart glasses. In: 2015 IEEE International Conference on Consumer Electronics (ICCE), pp. 585–586, Taipei, Taiwan (2015)

    Google Scholar 

  66. Li, S., Ashok, A., Zhang, Y., Xu, C., Lindqvist, J., Gruteser, M.: Whose move is it anyway? Authenticating smart wearable devices using unique head movement patterns. In: 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 1–9, Sydney, Australia (2016)

    Google Scholar 

  67. Kollee, B., Kratz, S., Dunnigan, A.: Exploring gestural interaction in smart spaces using head mounted devices with ego-centric sensing. In: Proceedings of the 2Nd ACM Symposium on Spatial User Interaction, SUI ’14, pp. 40–49, Honolulu, Hawaii, USA (2014)

    Google Scholar 

  68. Myers, B.A.: A brief history of human-computer interaction technology. Interactions 5(2), 44–54 (1998)

    Article  Google Scholar 

  69. Nirjon, S., Gummeson, J., Gelb, D., Kim, K.H.: Typingring: a wearable ring platform for text input. In: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys ’15, pp. 227–239, Florence, Italy (2015)

    Google Scholar 

  70. Ens, B., Byagowi, A., Han, T., Hincapié-Ramos, J.D. and Irani, P.: Combining ring input with hand tracking for precise, natural interaction with spatial analytic interfaces. In: Proceedings of the 2016 Symposium on Spatial User Interaction, SUI ’16, pp. 99–102, Tokyo, Japan (2016)

    Google Scholar 

  71. McCall, R., Martin, B., Popleteev, A., Louveton, N., Engel, T.: Text entry on smart glasses. In: 2015 8th International Conference on Human System Interaction (HSI), pp. 195–200, Warsaw, Poland, (2015)

    Google Scholar 

  72. Yi, X., Yu, C., Zhang, M., Gao, S., Sun, K., Shi, Y.: Atk: enabling ten-finger freehand typing in air based on 3d hand tracking data. In: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST ’15, pp. 539–548, Charlotte, NC, USA (2015)

    Google Scholar 

  73. Dunlop, M.D., Durga, N., Motaparti, S., Dona, P., Medapuram, V.: Qwerth: an optimized semi-ambiguous keyboard design. In: Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services Companion, MobileHCI ’12, pp. 23–28, San Francisco, California, USA. ACM, New York (2012)

    Google Scholar 

  74. Dunlop, M., Levine, J.: Multidimensional pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, pp. 2669–2678, Austin, Texas, USA. ACM, New York (2012)

    Google Scholar 

  75. Smith, B.A., Bi, X., Zhai, S.: Optimizing touchscreen keyboards for gesture typing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pages 3365–3374, Seoul, Republic of Korea. ACM, New York (2015)

    Google Scholar 

  76. Gong, J., Xu, Z., Guo, Q., Seyed, T., Chen, X.A., Bi, X., Yang, X.D.: Wristext: one-handed text entry on smartwatch using wrist gestures. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pages 181:1–181:14, Montreal QC, Canada. ACM (2018)

    Google Scholar 

  77. Mizobuchi, S., Terasaki, S., Keski-Jaskari, T., Nousiainen, J., Ryynanen, M., Silfverberg, M.: Making an impression: force-controlled pen input for handheld devices. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’05, pp. 1661–1664, Portland, OR, USA. ACM, New York (2005)

    Google Scholar 

  78. Wilson, G., Stewart, C., Brewster, S.A.: Pressure-based menu selection for mobile devices. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, MobileHCI ’10, pp. 181–190, Lisbon, Portugal. ACM, New York (2010)

    Google Scholar 

  79. Yin, J., Ren, X., Zhai, S.: Pen pressure control in trajectory-based interaction. Behav. Inform. Technol. 29, 137–148 (2010)

    Article  Google Scholar 

  80. Hsiu, M.-C., Huang, D.-Y., Chen, C.-A., Lin, Y.-C., Hung, Y.-P., Yang, D.-N., Chen, M.: Forceboard: using force as input technique on size-limited soft keyboard. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, MobileHCI ’16, pp. 599–604, Florence, Italy. ACM, New York (2016)

    Google Scholar 

  81. Zhong, M., Yu, C., Wang, Q., Xu, X., Shi, Y.: Forceboard: subtle text entry leveraging pressure. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pp. 528:1–528:10, Montreal QC, Canada (2018)

    Google Scholar 

  82. Hincapié-Ramos, J.D., Guo, X., Moghadasian, P., Irani, P.: Consumed endurance: A metric to quantify arm fatigue of mid-air interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, pp. 1063–1072, Toronto, Ontario, Canada. ACM, New York (2014)

    Google Scholar 

  83. Sujin Jang, Wolfgang Stuerzlinger, Satyajit Ambike, and Karthik Ramani. Modeling cumulative arm fatigue in mid-air interaction based on perceived exertion and kinetics of arm motion. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, pp. 3328–3339, Denver, Colorado, USA. ACM, New York (2017)

    Google Scholar 

  84. Vernier, F., Nigay, L.: A framework for the combination and characterization of output modalities. In: Proceedings of the 7th International Conference on Design, Specification, and Verification of Interactive Systems, DSV-IS’00, pp. 35–50, Limerick, Ireland. Springer, Berlin (2001)

    Google Scholar 

  85. Sobottka, K., Pitas, I.: A novel method for automatic face segmentation, facial feature extraction and tracking. Signal Process. Image Commun. 12(3), 263–281 (1998)

    Article  Google Scholar 

  86. Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables, vol. 55. Courier Dover Publications (1972)

    Google Scholar 

  87. Lee, L.H., Yung Lam, K., Yau, Y.P., Braud, T., Hui, P.: Hibey: hide the keyboard in augmented reality. In: 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom, pp. 1–10 (2019)

    Google Scholar 

  88. Bi, X., Smith, B.A., Zhai, S.: Quasi-qwerty soft keyboard optimization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, pp. 283–286, Atlanta, Georgia, USA (2010)

    Google Scholar 

  89. Gong, J., Tarasewich, P.: Alphabetically constrained keypad designs for text entry on mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’05, pp. 211–220, Portland, Oregon, USA. ACM, New York (2005)

    Google Scholar 

  90. Walmsley, W.S., Xavier Snelgrove, W., Truong, K.N.: Disambiguation of imprecise input with one-dimensional rotational text entry. ACM Trans. Comput.-Hum. Interact. 21(1), 4:1–4:40 (2014)

    Google Scholar 

  91. Bi, X., Zhai, S.: Bayesian touch: A statistical criterion of target selection with finger touch. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST ’13, pp. 51–60, St. Andrews, Scotland, United Kingdom (2013)

    Google Scholar 

  92. Goodman, J., Venolia, G., Steury, K., Parker, C.: Language modeling for soft keyboards. In: Proceedings of the 7th International Conference on Intelligent User Interfaces, IUI ’02, pp. 194–195, San Francisco, California, USA (2002)

    Google Scholar 

  93. Azenkot, S., Zhai, S.: Touch behavior with different postures on soft smartphone keyboards. In: Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services Companion, MobileHCI ’12, pp. 251–260, San Francisco, California, USA (2012)

    Google Scholar 

  94. Yu, C., Gu, Y., Yang, Z., Yi, X., Luo, H., Shi, Y.: Tap, dwell or gesture? Exploring head-based text entry techniques for HMDs. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, pp. 4479–4488, Denver, Colorado, USA. ACM, New York (2017)

    Google Scholar 

  95. Peshock, A., Duvall, J., Dunne, L.E.: Argot: a wearable one-handed keyboard glove. In: Proceedings of the 2014 ACM International Symposium on Wearable Computers: Adjunct Program, ISWC ’14 Adjunct, pp. 87–92, Seattle, Washington. ACM, New York (2014)

    Google Scholar 

  96. Zhai, S., Hunter, M., Smith, B.A.: The metropolis keyboard - an exploration of quantitative techniques for virtual keyboard design. In: Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology, UIST ’00, pp. 119–128, San Diego, California, USA. ACM, New York (2000)

    Google Scholar 

  97. Lu, Y., Yu, C., Yi, X., Shi, Y., Zhao, S.: Blindtype: eyes-free text entry on handheld touchpad by leveraging thumb’s muscle memory. In: ACM Interactive Mobile Wearable Ubiquitous Technology, vol. 1, pp. 18:1–18:24, Maul, Hawaii, USA. ACM, New York (2017)

    Google Scholar 

  98. Hwang, S., Lee, G.: Qwerty-like 3 × 4 keypad layouts for mobile phone. In: CHI ’05 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’05, pp. 1479–1482, Portland, OR, USA. ACM, New York (2005)

    Google Scholar 

  99. Wong, P.C., Zhu, K., Fu, H.: Fingert9: leveraging thumb-to-finger interaction for same-side-hand text entry on smartwatches. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pp. 178:1–178:10, Montreal QC, Canada. ACM, New York (2018)

    Google Scholar 

  100. Yu, C., Sun, K., Zhong, M., Li, X., Zhao, P., Shi, Y.: One-dimensional handwriting: Inputting letters and words on smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, pp. 71–82, San Jose, California, USA. ACM, New York (2016)

    Google Scholar 

  101. Grossman, T., Chen, X.A., Fitzmaurice, G.: Typing on glasses: adapting text entry to smart eyewear. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’15, pp. 144–152, Copenhagen, Denmark (2015)

    Google Scholar 

  102. Lee, L.H., Zhu, Y., Yau, Y.P., Braud, T., Su, X., Hui, P.: One-thumb text acquisition on force-assisted miniature interfaces for mobile headsets. In: 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom, pp. 1–10, (2020)

    Google Scholar 

Download references

Acknowledgements

This research has been supported in part by project 16214817 from the Research Grants Council of Hong Kong, and the 5GEAR project from the Academy of Finland ICT 2023 programme.

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea

    Lik-Hang Lee

  2. Division of Integrative Systems and Design, Hong Kong University of Science and Technology, Hong Kong, China

    Tristan Braud

  3. Computational Media and Arts Thrust, Hong Kong University of Science and Technology, Hong Kong, China

    Pan Hui

  4. Division of Integrative Systems and Design, Hong Kong University of Science and Technology, Hong Kong, China

    Pan Hui

Authors
  1. Lik-Hang Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tristan Braud
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pan Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Lik-Hang Lee or Tristan Braud .

Editor information

Editors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Andrew Yeh Ching Nee

  2. Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore

    Soh Khim Ong

Section Editor information

    Rights and permissions

    Reprints and permissions

    Copyright information

    © 2023 Springer Nature Switzerland AG

    About this chapter

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this chapter

    Lee, LH., Braud, T., Hui, P. (2023). Embodied Interaction on Constrained Interfaces for Augmented Reality. In: Nee, A.Y.C., Ong, S.K. (eds) Springer Handbook of Augmented Reality. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-030-67822-7_10

    Download citation

    • DOI: https://doi.org/10.1007/978-3-030-67822-7_10

    • Published:

    • Publisher Name: Springer, Cham

    • Print ISBN: 978-3-030-67821-0

    • Online ISBN: 978-3-030-67822-7

    • eBook Packages: Computer ScienceComputer Science (R0)

    Publish with us

    Policies and ethics

    Search

    Navigation