Michael Kintscher
Jinbin Huang
ABSTRACT
Augmented reality (AR) provides a significant opportunity to improve collaboration between co-located team members jointly analyzing data visualizations, but existing rigorous studies are lacking. We present a novel method for qualitatively encoding the positions of co-located users collaborating with head-mounted displays (HMDs) to assist in reliably analyzing collaboration styles and behaviors. We then perform a user study on the collaborative behaviors of multiple, co-located synchronously collaborating users in AR to demonstrate this method in practice and contribute to the shortfall of such studies in the existing literature. Pairs of users performed analysis tasks on several data visualizations using both AR and traditional desktop displays. To provide a robust evaluation, we collected several types of data, including software logging of participant positioning, qualitative analysis of video recordings of participant sessions, and pre- and post-study questionnaires including the NASA TLX survey. Our results suggest that the independent viewports of AR headsets reduce the need to verbally communicate about navigating around the visualization and encourage face-to-face and non-verbal communication. Our novel positional encoding method also revealed the overlap of task and communication spaces vary based on the needs of the collaborators.
- Cédric Bach and Dominique L Scapin. 2004. Obstacles and perspectives for evaluating mixed reality systems usability. In Acte du Workshop MIXER, IUI-CADUI, Vol. 4. Citeseer.Google Scholar
- Kevin M Baird and Woodrow Barfield. 1999. Evaluating the effectiveness of augmented reality displays for a manual assembly task. Virtual Reality 4, 4 (1999), 250–259.Google ScholarDigital Library
- Daniel Belcher, Mark Billinghurst, SE Hayes, and Randy Stiles. 2003. Using augmented reality for visualizing complex graphs in three dimensions. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. IEEE, 84–93.Google ScholarCross Ref
- Mark Billinghurst and Hirokazu Kato. 1999. Collaborative mixed reality. In Proceedings of the First International Symposium on Mixed Reality. 261–284.Google ScholarCross Ref
- Mark Billinghurst, Hirokazu Kato, Kiyoshi Kiyokawa, Daniel Belcher, and Ivan Poupyrev. 2002. Experiments with face-to-face collaborative AR interfaces. Virtual Reality 6, 3 (2002), 107–121.Google ScholarCross Ref
- Matthew Brehmer and Tamara Munzner. 2013. A multi-level typology of abstract visualization tasks. IEEE transactions on visualization and computer graphics 19, 12 (2013), 2376–2385.Google ScholarDigital Library
- Wolfgang Broll, Irma Lindt, Jan Ohlenburg, Michael Wittkämper, Chunrong Yuan, Thomas Novotny, Chiron Mottram, Andreas Strothmann, 2004. Arthur: A collaborative augmented environment for architectural design and urban planning. JVRB-Journal of Virtual Reality and Broadcasting 1, 1 (2004).Google Scholar
- Wolfgang Büschel, Patrick Reipschläger, Ricardo Langner, and Raimund Dachselt. 2017. Investigating the use of spatial interaction for 3d data visualization on mobile devices. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces. 62–71.Google ScholarDigital Library
- Jeffrey W Chastine, Kristine Nagel, Ying Zhu, and Luca Yearsovich. 2007. Understanding the design space of referencing in collaborative augmented reality environments. In Proceedings of graphics interface 2007. 207–214.Google ScholarDigital Library
- Arindam Dey, Mark Billinghurst, Robert W Lindeman, and J Swan. 2018. A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Frontiers in Robotics and AI 5 (2018), 37.Google ScholarCross Ref
- Suyang Dong, Amir H Behzadan, Feng Chen, and Vineet R Kamat. 2013. Collaborative visualization of engineering processes using tabletop augmented reality. Advances in Engineering Software 55 (2013), 45–55.Google ScholarDigital Library
- Andreas Dünser, Raphaël Grasset, and Mark Billinghurst. 2008. A survey of evaluation techniques used in augmented reality studies. Human Interface Technology Laboratory New Zealand.Google Scholar
- Ardita Dylgjeri and Ledia Kazazi. 2013. Deixis in modern linguistics and outside. Academic Journal of Interdisciplinary Studies 2, 4 (2013), 87–87.Google Scholar
- Rob Edsall and Kelli L Larson. 2006. Decision making in a virtual environment: Effectiveness of a semi-immersive “decision theater” in understanding and assessing human-environment interactions. In Proceedings of AutoCarto, Vol. 6. 25–28.Google Scholar
- Clarence A Ellis, Simon J Gibbs, and Gail Rein. 1991. Groupware: some issues and experiences. Commun. ACM 34, 1 (1991), 39–58.Google ScholarDigital Library
- Niklas Elmqvist, Pierre Dragicevic, and Jean-Daniel Fekete. 2008. Rolling the dice: Multidimensional visual exploration using scatterplot matrix navigation. IEEE transactions on Visualization and Computer Graphics 14, 6 (2008), 1539–1148.Google Scholar
- Charles J Fillmore. 1966. Deictic categories in the semantics of’come’. Foundations of language (1966), 219–227.Google Scholar
- Susan R. Fussell, Leslie D. Setlock, and Elizabeth M. Parker. 2003. Where Do Helpers Look? Gaze Targets during Collaborative Physical Tasks. In CHI ’03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI EA ’03). Association for Computing Machinery, New York, NY, USA, 768–769. https://doi.org/10.1145/765891.765980Google ScholarDigital Library
- Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004. Action as language in a shared visual space. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. 487–496.Google ScholarDigital Library
- Edwin L Hutchins, James D Hollan, and Donald A Norman. 1985. Direct manipulation interfaces. Human–computer interaction 1, 4 (1985), 311–338.Google ScholarDigital Library
- Kori Marie Inkpen. 1997. Adapting the human-computer interface to support collaborative learning environments for children. Ph. D. Dissertation. University of British Columbia.Google Scholar
- Petra Isenberg, Danyel Fisher, Sharoda A Paul, Meredith Ringel Morris, Kori Inkpen, and Mary Czerwinski. 2011. Co-located collaborative visual analytics around a tabletop display. IEEE Transactions on visualization and Computer Graphics 18, 5 (2011), 689–702.Google ScholarDigital Library
- Hiroshi Ishii, Minoru Kobayashi, and Kazuho Arita. 1994. Iterative design of seamless collaboration media. Commun. ACM 37, 8 (1994), 83–97.Google ScholarDigital Library
- Robert Johansen, David Sibbet, Suzyan Benson, Alixia Martin, Robert Mittman, and Paul Saffo. 1991. Leading business teams: How teams can use technology and group process tools to enhance performance. Addison-Wesley Longman Publishing Co., Inc.Google ScholarDigital Library
- Beatrice John, D Lang, Henrik von Wehrden, Ruediger John, and Arnim Wiek. 2018. Advancing decision-visualization environments: Empirically informed design guidelines.Google Scholar
- Kiyoshi Kiyokawa, Mark Billinghurst, Sohan E Hayes, Anoop Gupta, Yuki Sannohe, and Hirokazu Kato. 2002. Communication behaviors of co-located users in collaborative AR interfaces. In Proceedings. International Symposium on Mixed and Augmented Reality. IEEE, 139–148.Google ScholarCross Ref
- Robert E Kraut, Susan R Fussell, and Jane Siegel. 2003. Visual information as a conversational resource in collaborative physical tasks. Human–computer interaction 18, 1-2 (2003), 13–49.Google Scholar
- Robert E Kraut, Darren Gergle, and Susan R Fussell. 2002. The use of visual information in shared visual spaces: Informing the development of virtual co-presence. In Proceedings of the 2002 ACM conference on Computer supported cooperative work. 31–40.Google ScholarDigital Library
- Robert E Kraut, Mark D Miller, and Jane Siegel. 1996. Collaboration in performance of physical tasks: Effects on outcomes and communication. In Proceedings of the 1996 ACM conference on Computer supported cooperative work. 57–66.Google ScholarDigital Library
- Heidi Lam, Enrico Bertini, Petra Isenberg, Catherine Plaisant, and Sheelagh Carpendale. 2011. Empirical studies in information visualization: Seven scenarios. IEEE transactions on visualization and computer graphics 18, 9 (2011), 1520–1536.Google Scholar
- Matthew Lombard and Theresa Ditton. 1997. At the heart of it all: The concept of presence. Journal of computer-mediated communication 3, 2 (1997), JCMC 321.Google Scholar
- Tim Marsh. 2003. Staying there: an activity-based approach to narrative design and evaluation as an antidote to virtual corpsing. In Being there: Concepts, effects and measurement of user presence in synthetic environments, studies on new technologies and practices in communication series. Citeseer.Google Scholar
- Andreas Mathisen, Tom Horak, Clemens Nylandsted Klokmose, Kaj Grønbæk, and Niklas Elmqvist. 2019. InsideInsights: Integrating Data-Driven Reporting in Collaborative Visual Analytics. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 649–661.Google Scholar
- Mark McGill, Daniel Boland, Roderick Murray-Smith, and Stephen Brewster. 2015. A dose of reality: Overcoming usability challenges in vr head-mounted displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 2143–2152.Google Scholar
- Microsoft. 2020. HoloLens 2-Overview, Features, and Specs: Microsoft HoloLens. https://www.microsoft.com/en-us/hololens/hardware. https://www.microsoft.com/en-us/hololens/hardware.Google Scholar
- Jens Müller, Johannes Zagermann, Jonathan Wieland, Ulrike Pfeil, and Harald Reiterer. 2019. A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices. In Proceedings of Mensch und Computer 2019. 399–410.Google ScholarDigital Library
- Tamara Munzner. 2014. Visualization analysis and design. CRC press.Google Scholar
- Andrew YC Nee, SK Ong, George Chryssolouris, and Dimitris Mourtzis. 2012. Augmented reality applications in design and manufacturing. CIRP annals 61, 2 (2012), 657–679.Google Scholar
- Jiazhi Ou, Lui Min Oh, Jie Yang, and Susan R. Fussell. 2005. Effects of Task Properties, Partner Actions, and Message Content on Eye Gaze Patterns in a Collaborative Task. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Portland, Oregon, USA) (CHI ’05). Association for Computing Machinery, New York, NY, USA, 231–240. https://doi.org/10.1145/1054972.1055005Google ScholarDigital Library
- Kyoung S Park, Abhinav Kapoor, and Jason Leigh. 2000. Lessons learned from employing multiple perspectives in a collaborative virtual environment for visualizing scientific data. In Proceedings of the Third International Conference on Collaborative Virtual Environments. 73–82.Google ScholarDigital Library
- Dheva Raja, Doug Bowman, John Lucas, and Chris North. 2004. Exploring the benefits of immersion in abstract information visualization. In Proc. Immersive Projection Technology Workshop. 61–69.Google Scholar
- Dieter Schmalstieg, Anton Fuhrmann, Gerd Hesina, Zsolt Szalavári, L Miguel Encarnaçao, Michael Gervautz, and Werner Purgathofer. 2002. The studierstube augmented reality project. Presence: Teleoperators & Virtual Environments 11, 1 (2002), 33–54.Google ScholarDigital Library
- Mickael Sereno, Xiyao Wang, Lonni Besancon, Michael J Mcguffin, and Tobias Isenberg. 2020. Collaborative Work in Augmented Reality: A Survey. IEEE Transactions on Visualization and Computer Graphics (2020), 1–1. https://doi.org/10.1109/TVCG.2020.3032761Google ScholarDigital Library
- Ben Shneiderman. 2003. The eyes have it: A task by data type taxonomy for information visualizations. In The craft of information visualization. Elsevier, 364–371.Google Scholar
- Rebecca R Springmeyer, Meera M Blattner, and Nelson L Max. 1992. A characterization of the scientific data analysis process. In Proceedings of the 3rd conference on Visualization’92. IEEE Computer Society Press, 235–242.Google ScholarCross Ref
- Zsolt Szalavári, Dieter Schmalstieg, Anton Fuhrmann, and Michael Gervautz. 1998. “Studierstube”: An environment for collaboration in augmented reality. Virtual Reality 3, 1 (1998), 37–48.Google ScholarDigital Library
- Arthur Tang, Frank Biocca, and Lynette Lim. 2004. Comparing differences in presence during social interaction in augmented reality versus virtual reality environments: An exploratory study. Proceedings of PRESENCE (2004), 204–208.Google Scholar
- Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems. 73–80.Google ScholarDigital Library
- Anthony Tang, Melanie Tory, Barry Po, Petra Neumann, and Sheelagh Carpendale. 2006. Collaborative coupling over tabletop displays. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 1181–1190.Google ScholarDigital Library
- Melanie Tory, Arthur E Kirkpatrick, M Stella Atkins, and Torsten Moller. 2005. Visualization task performance with 2D, 3D, and combination displays. IEEE transactions on visualization and computer graphics 12, 1 (2005), 2–13.Google Scholar
- April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How data scientists use computational notebooks for real-time collaboration. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–30.Google ScholarDigital Library
- Xiangyu Wang and Phillip S Dunston. 2011. Comparative effectiveness of mixed reality-based virtual environments in collaborative design. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 41, 3 (2011), 284–296.Google ScholarDigital Library
- Duncan J Watts and Steven H Strogatz. 1998. Collective dynamics of ‘small-world’ networks. nature 393, 6684 (1998), 440–442.Google Scholar
- Matt Whitlock, Stephen Smart, and Danielle Albers Szafir. 2020. Graphical Perception for Immersive Analytics. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 616–625.Google Scholar
- Jian Zhao, Michael Glueck, Petra Isenberg, Fanny Chevalier, and Azam Khan. 2017. Supporting handoff in asynchronous collaborative sensemaking using knowledge-transfer graphs. IEEE transactions on visualization and computer graphics 24, 1 (2017), 340–350.Google Scholar
Index Terms
- Measuring and Comparing Collaborative Visualization Behaviors in Desktop and Augmented Reality Environments
Recommendations
Collaborative Augmented Reality: Multi-user Interaction in Urban Simulation
IVIC '09: Proceedings of the 1st International Visual Informatics Conference on Visual Informatics: Bridging Research and PracticeAugmented reality (AR) environment allows user or multi-user to interact with 2D and 3D data. AR simply can provide a collaborative interactive AR environment for urban simulation, where users can interact naturally and intuitively. AR collaboration ...
A Qualitative Comparison Between Augmented and Virtual Reality Collaboration with Handheld Devices
MuC '19: Proceedings of Mensch und Computer 2019Handheld Augmented Reality (AR) displays offer a see-through option to create the illusion of virtual objects being integrated into the viewer's physical environment. Some AR display technologies also allow for the deactivation of the see-through option,...
SelectVisAR: Selective Visualisation of Virtual Environments in Augmented Reality
DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems ConferenceWhen establishing a visual connection between a virtual reality user and an augmented reality user, it is important to consider whether the augmented reality user faces a surplus of information. Augmented reality, compared to virtual reality, involves ...
Comments