Hemant Bhaskar Surale
Yu Jiang Tham
Brian A. Smith
ABSTRACT
Augmented Reality (AR) smartglasses are increasingly regarded as the next generation personal computing platform. However, there is a lack of understanding about how to design communication systems using them. We present ARcall, a novel Augmented Reality-based real-time communication system that enables an immersive, delightful, and privacy-preserving experience between a smartphone user and a smartglasses wearer. ARcall allows a remote friend (Friend) to send and project AR content to a smartglasses wearer (Wearer). The ARcall system was designed with the practical limits of existing AR glasses in mind, including shorter battery life and a reduced field of view. We conduct a qualitative evaluation of the three main components of ARcall: Drop-In, ARaction, and Micro-Chat. Our results provide novel insights for building future AR-based communication methods, including, the importance of context priming, user control over AR content placement, and the feeling of co-presence while conversing.
Supplemental Material
- Mark S Ackerman. 2000. The intellectual challenge of CSCW: The gap between social requirements and technical feasibility. Human–Computer Interaction 15, 2-3 (2000), 179–203.Google Scholar
Digital Library
- Mark S Ackerman and Scott D Mainwaring. 2005. Privacy issues and human-computer interaction. Computer 27, 5 (2005), 19–26.Google Scholar
- Alpha Exploration Co.2020. Clubhouse. https://www.joinclubhouse.com.Google Scholar
- Morgan G Ames, Janet Go, Joseph’Jofish’ Kaye, and Mirjana Spasojevic. 2010. Making love in the network closet: the benefits and work of family videochat. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. 145–154.Google ScholarDigital Library
- Pengcheng An, Ziqi Zhou, Qing Liu, Yifei Yin, Linghao Du, Da-Yuan Huang, and Jian Zhao. 2021. VibEmoji: Exploring User-authoring Multi-modal Emoticons in Social Communication. arXiv preprint arXiv:2112.13555(2021).Google Scholar
- Apple Inc.2019. SceneKit | Apple Developer Documentation. https://developer.apple.com/documentation/scenekit/.Google Scholar
- Apple Inc.2020. ARKit - Augmented Reality - Apple Developer. https://developer.apple.com/augmented-reality/arkit/.Google Scholar
- Apple Inc.2020. SpriteKit - Apple Developer. https://developer.apple.com/spritekit/.Google Scholar
- Uddipana Baishya and Carman Neustaedter. 2017. In Your Eyes: Anytime, Anywhere Video and Audio Streaming for Couples. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 84–97.Google ScholarDigital Library
- Mark Billinghurst, Alaeddin Nassani, and Carolin Reichherzer. 2014. Social panoramas: using wearable computers to share experiences. In SIGGRAPH Asia 2014 Mobile Graphics and Interactive Applications. 1–1.Google Scholar
- Taryn Bipat, Maarten Willem Bos, Rajan Vaish, and Andrés Monroy-Hernández. 2019. Analyzing the use of camera glasses in the wild. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–8.Google ScholarDigital Library
- Blade 2020. Vuzix Blade. https://www.vuzix.com/products/blade-smart-glasses.Google Scholar
- Ashley Carman. 2019. Snapchat launches another aging AR lens to lure people back to the app. https://www.theverge.com/2019/11/21/20974367/snapchat-time-machine-baby-aging-lens-ar-launch.Google Scholar
- Henriette Cramer, Mattias Rost, and Lars Erik Holmquist. 2011. Performing a check-in: emerging practices, norms and’conflicts’ in location-sharing using foursquare. In Proceedings of the 13th international conference on human computer interaction with mobile devices and services. 57–66.Google ScholarDigital Library
- Edward B Cutrell, Mary Czerwinski, and Eric Horvitz. 2000. Effects of instant messaging interruptions on computing tasks. In CHI’00 extended abstracts on Human factors in computing systems. 99–100.Google Scholar
- Fabien Danieau, Antoine Guillo, and Renaud Doré. 2017. Attention guidance for immersive video content in head-mounted displays. In 2017 IEEE Virtual Reality (VR). IEEE, 205–206.Google Scholar
- Tamara Denning, Zakariya Dehlawi, and Tadayoshi Kohno. 2014. In situ with bystanders of augmented reality glasses: Perspectives on recording and privacy-mediating technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2377–2386.Google ScholarDigital Library
- Nicholas Epley and Juliana Schroeder. 2014. Mistakenly seeking solitude.Journal of Experimental Psychology: General 143, 5 (2014), 1980.Google Scholar
- Facebook, Inc.2020. Instagram. https://instagram.com/.Google Scholar
- FBRooms 2020. Facebook Rooms. https://about.fb.com/news/2020/04/introducing-messenger-rooms/.Google Scholar
- Henry Fuchs, Gary Bishop, Kevin Arthur, Leonard McMillan, Ruzena Bajcsy, Sang Lee, Hany Farid, and Takeo Kanade. 1994. Virtual space teleconferencing using a sea of cameras. In Proc. First International Conference on Medical Robotics and Computer Assisted Surgery, Vol. 26.Google Scholar
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. In Touch with the Remote World: Remote Collaboration with Augmented Reality Drawings and Virtual Navigation. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology - VRST ’14. ACM Press, Edinburgh, Scotland, 197–205. https://doi.org/10.1145/2671015.2671016Google ScholarDigital Library
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. In touch with the remote world: Remote collaboration with augmented reality drawings and virtual navigation. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 197–205.Google ScholarDigital Library
- Google Glass 2019. Glass. https://developers.google.com/glass/develop/gdk/touch.Google Scholar
- Google Inc.2020. Firebase. https://firebase.google.com/.Google Scholar
- Graffity 2020. Graffity App. https://www.youtube.com/watch?v=DVy8SHpTxsA.Google Scholar
- Saul Greenberg and Carman Neustaedter. 2013. Shared living, experiences, and intimacy over video chat in long distance relationships. In Connecting families. Springer, 37–53.Google Scholar
- Daniel A Hashimoto, Roy Phitayakorn, Carlos Fernandez-del Castillo, and Ozanan Meireles. 2016. A blinded assessment of video quality in wearable technology for telementoring in open surgery: the Google Glass experience. Surgical endoscopy 30, 1 (2016), 372–378.Google Scholar
- Zhenyi He, Ruofei Du, and Ken Perlin. 2020. CollaboVR: A Reconfigurable Framework for Creative Collaboration in Virtual Reality. In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 542–554.Google Scholar
- Steven J Henderson and Steven K Feiner. 2011. Augmented reality in the psychomotor phase of a procedural task. In 2011 10th IEEE International Symposium on Mixed and Augmented Reality. IEEE, 191–200.Google ScholarDigital Library
- Tobias Höllerer, Steven Feiner, Tachio Terauchi, Gus Rashid, and Drexel Hallaway. 1999. Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system. Computers & Graphics 23, 6 (1999), 779–785.Google ScholarCross Ref
- Houseparty 2020. Houseparty. https://houseparty.com/.Google Scholar
- North Inc.2019. Focals by North. https://support.bynorth.com/.Google Scholar
- Snap Inc.2021. Next Generation Spectacles. https://www.spectacles.com/ca-en/new-spectacles/.Google Scholar
- Hyungeun Jo and Sungjae Hwang. 2013. Chili: Viewpoint Control and on-Video Drawing for Mobile Video Calls. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems on - CHI EA ’13. ACM Press, Paris, France, 1425. https://doi.org/10.1145/2468356.2468610Google ScholarDigital Library
- Hyungeun Jo and Sungjae Hwang. 2013. Chili: viewpoint control and on-video drawing for mobile video calls. In CHI’13 Extended Abstracts on Human Factors in Computing Systems. 1425–1430.Google Scholar
- Simon Jones and Eamonn O’Neill. 2011. Contextual dynamics of group-based sharing decisions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1777–1786.Google ScholarDigital Library
- Tejinder K Judge and Carman Neustaedter. 2010. Sharing conversation and sharing life: video conferencing in the home. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 655–658.Google ScholarDigital Library
- Joseph’Jofish’ Kaye. 2006. I just clicked to say I love you: rich evaluations of minimal communication. In CHI’06 extended abstracts on human factors in computing systems. 363–368.Google Scholar
- Ryan Kelly and Leon Watts. 2015. Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Experiences of technology appropriation: Unanticipated users, usage, circumstances, and design 2(2015).Google Scholar
- Sujay Khandekar, Joseph Higg, Yuanzhe Bian, Chae Won Ryu, Jerry O. Talton Iii, and Ranjitha Kumar. 2019. Opico: a study of emoji-first communication in a mobile social app. In Companion Proceedings of The 2019 World Wide Web Conference. 450–458.Google ScholarDigital Library
- Kibum Kim, John Bolton, Audrey Girouard, Jeremy Cooperstock, and Roel Vertegaal. 2012. Telehuman: effects of 3d perspective on gaze and pose estimation with a life-size cylindrical telepresence pod. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2531–2540.Google ScholarDigital Library
- Seungwon Kim, Gun Lee, Weidong Huang, Hayun Kim, Woontack Woo, and Mark Billinghurst. 2019. Evaluating the Combination of Visual Communication Cues for HMD-Based Mixed Reality Remote Collaboration. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19. ACM Press, Glasgow, Scotland Uk, 1–13. https://doi.org/10.1145/3290605.3300403Google ScholarDigital Library
- Seungwon Kim, Gun Lee, Weidong Huang, Hayun Kim, Woontack Woo, and Mark Billinghurst. 2019. Evaluating the combination of visual communication cues for HMD-based mixed reality remote collaboration. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–13.Google ScholarDigital Library
- Grzegorz Krawczyk, Karol Myszkowski, and Hans-Peter Seidel. 2005. Perceptual effects in real-time tone mapping. In Proceedings of the 21st spring conference on Computer graphics. 195–202.Google ScholarDigital Library
- Andrew L Kun, Hidde van der Meulen, and Christian P Janssen. 2019. Calling while driving using augmented reality: Blessing or curse?PRESENCE: Virtual and Augmented Reality 27, 1 (2019), 1–14.Google ScholarDigital Library
- Hideaki Kuzuoka, Toshio Kosuge, and Masatomo Tanaka. 1994. GestureCam: A video communication system for sympathetic remote collaboration. In Proceedings of the 1994 ACM conference on Computer supported cooperative work. 35–43.Google ScholarDigital Library
- Mikko Kytö, Ilyena Hirskyj-Douglas, and David McGookin. 2021. From Strangers to Friends: Augmenting Face-to-face Interactions with Faceted Digital Self-Presentations. In Augmented Humans Conference 2021. 192–203.Google ScholarDigital Library
- Scott Lederer, Anind K Dey, and Jennifer Mankoff. 2002. A conceptual model and a metaphor of everyday privacy in ubiquitous computing environments. Computer Science Division, University of California.Google Scholar
- Ryan Louie, Kapil Garg, Jennie Werner, Allison Sun, Darren Gergle, and Haoqi Zhang. 2021. Opportunistic Collective Experiences: Identifying Shared Situations and Structuring Shared Activities at Distance. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3(2021), 1–32.Google ScholarDigital Library
- Kodai Matsuhashi, Toshiki Kanamoto, and Atsushi Kurokawa. 2020. Thermal model and countermeasures for future smart glasses. Sensors 20, 5 (2020), 1446.Google ScholarCross Ref
- Jörg Müller, Tobias Langlotz, and Holger Regenbrecht. 2016. PanoVC: Pervasive telepresence using mobile phones. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, 1–10.Google ScholarCross Ref
- Bonnie A Nardi, Steve Whittaker, and Erin Bradner. 2000. Interaction and outeraction: instant messaging in action. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. 79–88.Google ScholarDigital Library
- Alaeddin Nassani, Hyungon Kim, Gun Lee, Mark Billinghurst, Tobias Langlotz, and Robert W. Lindeman. 2016. Augmented Reality Annotation for Social Video Sharing. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications on - SA ’16. ACM Press, Macau, 1–5. https://doi.org/10.1145/2999508.2999529Google ScholarDigital Library
- Alaeddin Nassani, Hyungon Kim, Gun Lee, Mark Billinghurst, Tobias Langlotz, and Robert W Lindeman. 2016. Augmented reality annotation for social video sharing. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications. 1–5.Google ScholarDigital Library
- Carman Neustaedter, Erick Oduor, Gina Venolia, and Tejinder K Judge. 2012. Moving beyond talking heads to shared experiences: the future of personal video communication. In Proceedings of the 17th ACM international conference on Supporting group work. 327–330.Google ScholarDigital Library
- Carman Neustaedter, Carolyn Pang, Azadeh Forghani, Erick Oduor, Serena Hillman, Tejinder K Judge, Michael Massimi, and Saul Greenberg. 2015. Sharing domestic life through long-term video connections. ACM Transactions on Computer-Human Interaction (TOCHI) 22, 1(2015), 1–29.Google ScholarDigital Library
- Carman Neustaedter, Jason Procyk, Anezka Chua, Azadeh Forghani, and Carolyn Pang. 2020. Mobile video conferencing for sharing outdoor leisure activities over distance. Human–Computer Interaction 35, 2 (2020), 103–142.Google Scholar
- Casey Newton. 2018. You can now build your own face filter for Snapchat. https://www.theverge.com/2018/4/17/17245008/snapchat-face-filter-creation-lens-studio-giphy-discover.Google Scholar
- Next-Gen Spectacles 2021. Snap Spectacles. https://www.spectacles.com/ca-en/new-spectacles.Google Scholar
- Nreal 2020. NREAL AI. https://www.nreal.ai/.Google Scholar
- Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2015. Virtual replicas for remote assistance in virtual and augmented reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. 405–415.Google ScholarDigital Library
- Kenton O’Hara, Alison Black, and Matthew Lipson. 2006. Everyday practices with mobile video telephony. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 871–880.Google ScholarDigital Library
- Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L Davidson, Sameh Khamis, Mingsong Dou, 2016. Holoportation: Virtual 3d teleportation in real-time. In Proceedings of the 29th annual symposium on user interface software and technology. 741–754.Google ScholarDigital Library
- Antti Oulasvirta, Sakari Tamminen, Virpi Roto, and Jaana Kuorelahti. 2005. Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI. In Proceedings of the SIGCHI conference on Human factors in computing systems. 919–928.Google ScholarDigital Library
- Kevin P Pfeil, Neeraj Chatlani, Joseph J LaViola Jr, and Pamela Wisniewski. 2021. Bridging the Socio-Technical Gaps in Body-worn Interpersonal Live-Streaming Telepresence through a Critical Review of the Literature. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–39.Google ScholarDigital Library
- Bastian Pfleging, Stefan Schneegass, and Albrecht Schmidt. 2013. Exploring user expectations for context and road video sharing while calling and driving. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 132–139.Google ScholarDigital Library
- Thammathip Piumsomboon, Gun A Lee, Jonathon D Hart, Barrett Ens, Robert W Lindeman, Bruce H Thomas, and Mark Billinghurst. 2018. Mini-me: An adaptive avatar for mixed reality remote collaboration. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–13.Google ScholarDigital Library
- Thammathip Piumsomboon, Youngho Lee, Gun A. Lee, Arindam Dey, and Mark Billinghurst. 2017. Empathic Mixed Reality: Sharing What You Feel and Interacting with What You See. In 2017 International Symposium on Ubiquitous Virtual Reality (ISUVR). IEEE, Nara, Japan, 38–41. https://doi.org/10.1109/ISUVR.2017.20Google Scholar
- Jason Procyk, Carman Neustaedter, Carolyn Pang, Anthony Tang, and Tejinder K Judge. 2014. Exploring video streaming in public settings: shared geocaching over distance using mobile video chat. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2163–2172.Google ScholarDigital Library
- Xukan Ran, Carter Slocum, Maria Gorlatova, and Jiasi Chen. 2019. ShareAR: Communication-efficient multi-user mobile augmented reality. In Proceedings of the 18th ACM Workshop on Hot Topics in Networks. 109–116.Google ScholarDigital Library
- Jan Ole Rixen, Teresa Hirzle, Mark Colley, Yannick Etzel, Enrico Rukzio, and Jan Gugenheimer. 2021. Exploring Augmented Visual Alterations in Interpersonal Communication. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11.Google ScholarDigital Library
- Rokid Inc. 2019. Rokid Glass. https://glass.rokid.com/en/.Google Scholar
- Bektur Ryskeldiev, Michael Cohen, and Jens Herder. 2018. Streamspace: Pervasive mixed reality telepresence for remote collaboration on mobile devices. Journal of Information Processing 26 (2018), 177–185.Google ScholarCross Ref
- Emily Schildt, Martin Leinfors, and Louise Barkhuus. 2016. Communication, coordination and awareness around continuous location sharing. In Proceedings of the 19th International Conference on Supporting Group Work. 257–265.Google ScholarDigital Library
- Sinch. 2020. Video Calling - Get face to face with our simple APIs and SDKs. https://www.sinch.com/products/apis/calling/video/.Google Scholar
- Smoodji 2018. Smoodji Augmented Reality Application. https://www.octosense.com/project/smoodji-augmented-reality-messaging/.Google Scholar
- Spectacles 2019. Snap Focals. https://www.spectacles.com/ca-en/learn/.Google Scholar
- Balasaravanan Thoravi Kumaravel, Fraser Anderson, George Fitzmaurice, Bjoern Hartmann, and Tovi Grossman. 2019. Loki: Facilitating remote instruction of physical tasks using bi-directional mixed-reality telepresence. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 161–174.Google ScholarDigital Library
- Qianli Xu, Michal Mukawa, Liyuan Li, Joo Hwee Lim, Cheston Tan, Shue Ching Chia, Tian Gan, and Bappaditya Mandal. 2015. Exploring users’ attitudes towards social interaction assistance on Google Glass. In Proceedings of the 6th Augmented Human international conference. 9–12.Google ScholarDigital Library
Recommendations
Exploring Immersive Interpersonal Communication via AR
CSCWA central challenge of social computing research is to enable people to communicate expressively with each other remotely. Augmented reality has great promise for expressive communication since it enables communication beyond texts and photos and towards ...
Exhibition approach using an AR and VR pillar
SA '17: SIGGRAPH Asia 2017 Mobile Graphics & Interactive ApplicationsThis demonstration presents a development of an Augmented Reality (AR) and Virtual Reality (AR) pillar, a novel approach for showing AR and VR content in a public setting. A pillar in a public exhibition venue was converted to a four-sided AR and VR ...
Requirements for Virtualization of AR Displays within VR Environments
Proceedings, Part I, of the 6th International Conference on Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments - Volume 8525Everybody has been talking about new emerging products in augmented reality AR and their potential to enhance our daily life and work. The AR technology has been around for quite a while and various use cases have been thought and tested. Clearly, the ...
Comments