Jonathan Wieland
ABSTRACT
Mobile devices such as phones and tablets have become the most prevalent AR devices and are applied in various domains, including architecture, entertainment, and education. However, due to device-specific characteristics, designing and evaluating handheld AR interactions can be especially challenging as handheld AR displays provide 1) limited input options, 2) a narrow camera field of view, and 3) restricted context awareness. To address these challenges with design recommendations and research implications, the dissertation follows a mixed-methods approach with three research questions (RQs): On the one hand, specific aspects of fundamental 3D object manipulation tasks (RQ1) and awareness and discoverability (RQ2) are explored and evaluated in controlled lab studies. For increased ecological validity, the developed interaction concepts are also investigated in public interactive exhibitions. These studies then inform the creation of a framework for designing and evaluating handheld AR experiences using VR simulations of the interaction context (RQ3).
- Ronald T. Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoper. Virtual Environ. 6, 4 (Aug. 1997), 355–385. https://doi.org/10.1162/pres.1997.6.4.355Google Scholar
Digital Library
- Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends® in Human–Computer Interaction 8, 2-3 (2015), 73–272. https://doi.org/10.1561/1100000049Google ScholarDigital Library
- Stefano Burigat and Luca Chittaro. 2007. Navigation in 3D virtual environments: Effects of user experience and location-pointing navigation aids. International Journal of Human-Computer Studies 65, 11 (2007), 945–958. https://doi.org/10.1016/j.ijhcs.2007.07.003Google ScholarDigital Library
- Wolfgang Büschel, Anke Lehmann, and Raimund Dachselt. 2021. MIRIA: A Mixed Reality Toolkit for the In-Situ Visualization and Analysis of Spatio-Temporal Interaction Data. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 470, 15 pages. https://doi.org/10.1145/3411764.3445651Google ScholarDigital Library
- Luca Chittaro and Stefano Burigat. 2004. 3D Location-Pointing as a Navigation Aid in Virtual Environments. In Proceedings of the Working Conference on Advanced Visual Interfaces (Gallipoli, Italy) (AVI ’04). Association for Computing Machinery, New York, NY, USA, 267–274. https://doi.org/10.1145/989863.989910Google ScholarDigital Library
- J. G. Grandi, H. G. Debarba, I. Bemdt, L. Nedel, and A. Maciel. 2018. Design and Assessment of a Collaborative 3D Interaction Technique for Handheld Augmented Reality. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 49–56. https://doi.org/10.1109/VR.2018.8446295Google ScholarCross Ref
- J. G. Grandi, H. G. Debarba, and A. Maciel. 2019. Characterizing Asymmetric Collaborative Interactions in Virtual and Augmented Realities. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 127–135. https://doi.org/10.1109/VR.2019.8798080Google ScholarCross Ref
- Uwe Gruenefeld, Abdallah El Ali, Wilko Heuten, and Susanne Boll. 2017. Visualizing Out-of-View Objects in Head-Mounted Augmented Reality. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (Vienna, Austria) (MobileHCI ’17). Association for Computing Machinery, New York, NY, USA, Article 81, 7 pages. https://doi.org/10.1145/3098279.3122124Google ScholarDigital Library
- Uwe Gruenefeld, Jonas Auda, Florian Mathis, Stefan Schneegass, Mohamed Khamis, Jan Gugenheimer, and Sven Mayer. 2022. VRception: Rapid Prototyping of Cross-Reality Systems in Virtual Reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 611, 15 pages. https://doi.org/10.1145/3491102.3501821Google ScholarDigital Library
- Sebastian Hubenschmid, Jonathan Wieland, Daniel Immanuel Fink, Andrea Batch, Johannes Zagermann, Niklas Elmqvist, and Harald Reiterer. 2022. ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 24, 20 pages. https://doi.org/10.1145/3491102.3517550Google ScholarDigital Library
- Sebastian Hubenschmid, Johannes Zagermann, Daniel Fink, Jonathan Wieland, Tiare Feuchtner, and Harald Reiterer. 2021. Towards Asynchronous Hybrid User Interfaces for Cross-Reality Interaction. In ISS’21 Workshop Proceedings: "Transitional Interfaces in Mixed and Cross-Reality: A new frontier?". https://doi.org/10.18148/kops/352-2-84mm0sggczq02Google ScholarCross Ref
- Joseph J. LaViola. 2017. 3D user interfaces: theory and practice (second ed.). Addison-Wesley, Boston.Google Scholar
- Jens Müller, Johannes Zagermann, Jonathan Wieland, Ulrike Pfeil, and Harald Reiterer. 2019. A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices. In Proceedings of Mensch Und Computer 2019 (Hamburg, Germany) (MuC’19). Association for Computing Machinery, New York, NY, USA, 399–410. https://doi.org/10.1145/3340764.3340773Google ScholarDigital Library
- Patrick Perea 2017. Halo3D: A Technique for Visualizing Off-Screen Points of Interest in Mobile Augmented Reality. Adjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017 3267 (2017), 170–175. https://doi.org/10.1109/ISMAR-Adjunct.2017.58Google ScholarDigital Library
- Patrick Perea, Denis Morand, and Laurence Nigay. 2019. Spotlight on Off-Screen Points of Interest in Handheld Augmented Reality: Halo-Based Techniques. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces (Daejeon, Republic of Korea) (ISS ’19). Association for Computing Machinery, New York, NY, USA, 43–54. https://doi.org/10.1145/3343055.3359719Google ScholarDigital Library
- Márcio S. Pinho, Doug A. Bowman, and Carla M.D.S. Freitas. 2002. Cooperative Object Manipulation in Immersive Virtual Environments: Framework and Techniques. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (Hong Kong, China) (VRST ’02). Association for Computing Machinery, New York, NY, USA, 171–178. https://doi.org/10.1145/585740.585769Google ScholarDigital Library
- Marcio S. Pinho, Doug A. Bowman, and Carla M. Dal Sasso Freitas. 2008. Cooperative object manipulation in collaborative virtual environments. Journal of the Brazilian Computer Society 14, 2 (01 06 2008), 53–67. https://doi.org/10.1007/BF03192559Google ScholarCross Ref
- Patrick Renner and Thies Pfeiffer. 2017. Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. In 2017 IEEE Symposium on 3D User Interfaces (3DUI). 186–194. https://doi.org/10.1109/3DUI.2017.7893338Google ScholarCross Ref
- Kai Riege, T. Holtkamper, G. Wesche, and B. Frohlich. 2006. The Bent Pick Ray: An Extended Pointing Technique for Multi-User Interaction. In 3D User Interfaces (3DUI’06). 62–65. https://doi.org/10.1109/VR.2006.127Google ScholarDigital Library
- Roy A. Ruddle, Justin C. D. Savage, and Dylan M. Jones. 2002. Symmetric and Asymmetric Action Integration during Cooperative Object Manipulation in Virtual Environments. ACM Trans. Comput.-Hum. Interact. 9, 4 (Dec. 2002), 285–308. https://doi.org/10.1145/586081.586084Google ScholarDigital Library
- Roy A. Ruddle, Justin C. D. Savage, and Dylan M. Jones. 2003. Levels of Control During a Collaborative Carrying Task. Presence: Teleoperators and Virtual Environments 12, 2 (2003), 140–155. https://doi.org/10.1162/105474603321640914Google ScholarDigital Library
- Dieter Schmalstieg and Tobias Höllerer. 2016. Augmented Reality: Principles and Practice. Addison–Wesley Professional.Google Scholar
- Iris Seidinger and Jens Grubert. 2016. 3D Character Customization for Non-Professional Users in Handheld Augmented Reality. In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). 129–134. https://doi.org/10.1109/ISMAR-Adjunct.2016.0059Google ScholarCross Ref
- Teresa Siu and Valeria Herskovic. 2013. SidebARs: Improving Awareness of off-Screen Elements in Mobile Augmented Reality. In Proceedings of the 2013 Chilean Conference on Human - Computer Interaction (Temuco, Chile) (ChileCHI ’13). Association for Computing Machinery, New York, NY, USA, 36–41. https://doi.org/10.1145/2535597.2535608Google ScholarDigital Library
- Moritz Skowronski, Jonathan Wieland, Marcel Borowski, Daniel Fink, Carla Gröschel, Daniel Klinkhammer, and Harald Reiterer. 2018. Blended Museum: The Interactive Exhibition "Rebuild Palmyra?". In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia (Cairo, Egypt) (MUM 2018). Association for Computing Machinery, New York, NY, USA, 529–535. https://doi.org/10.1145/3282894.3289746Google ScholarDigital Library
- Rishi Vanukuru, Amarnath Murugan, and Jayesh Pillai. 2020. Dual Phone AR: Exploring the Use of Phones as Controllers for Mobile Augmented Reality. In Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology(VRST ’20). Association for Computing Machinery, New York, NY, USA, Article 52. https://doi.org/10.1145/3385956.3422113Google ScholarDigital Library
- Philipp Wacker, Oliver Nowak, Simon Voelker, and Jan Borchers. 2019. ARPen: Mid-Air Object Manipulation Techniques for a Bimanual AR System with Pen & Smartphone. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300849Google ScholarDigital Library
- Jonathan Wieland, Rudolf C. Hegemann Garcia, Harald Reiterer, and Tiare Feuchtner. 2022. Arrow, Bézier Curve, or Halos? – Comparing 3D Out-of-View Object Visualization Techniques for Handheld Augmented Reality. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 797–806. https://doi.org/10.1109/ISMAR55827.2022.00098Google ScholarCross Ref
- Jonathan Wieland, Johannes Zagermann, Jens Müller, and Harald Reiterer. 2021. Separation, Composition, or Hybrid? – Comparing Collaborative 3D Object Manipulation Techniques for Handheld Augmented Reality. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 403–412. https://doi.org/10.1109/ISMAR52148.2021.00057Google ScholarCross Ref
- Difeng Yu, Hai-Ning Liang, Kaixuan Fan, Heng Zhang, Charles Fleming, and Konstantinos Papangelis. 2020. Design and Evaluation of Visualization Techniques of Off-Screen and Occluded Targets in Virtual Reality Environments. IEEE Transactions on Visualization and Computer Graphics 26, 9 (2020), 2762–2774. https://doi.org/10.1109/TVCG.2019.2905580Google ScholarCross Ref
Index Terms
- Designing and Evaluating Interactions for Handheld AR
Recommendations
The Second Workshop on Multiple Input Modalities and Sensations for VR/AR Interactions (MIMSVAI)
UbiComp/ISWC '22 Adjunct: Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable ComputersWith the advance of VR/AR technology, more and more VR/AR applications are emerging and have been popular among new users. Interacting with virtual reality and augmented reality technologies will require the development of alternative input modalities ...
I’m in Control! Transferring Object Ownership Between Remote Users with Haptic Props in Virtual Reality
SUI '21: Proceedings of the 2021 ACM Symposium on Spatial User InteractionVirtual Reality (VR) remote collaboration is becoming more and more relevant in a wide range of scenarios, such as remote assistance or group work. A way to enhance the user experience is using haptic props that make virtual objects graspable. But ...
The First Workshop on Multiple Input Modalities and Sensations for VR/AR Interactions (MIMSVAI)
UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable ComputersWith the advance of VR/AR technology, more and more VR/AR applications are emerging and have been popular among new users. Users interact with both physical or virtual worlds through multiple input modalities and sensations to create an immersive VR/AR ...
Comments