Kazuma Takada
Jun Nishida
Shunichi Kasahara
ABSTRACT
In this paper, we propose ShadowClones, an interface that supports interactions in which a single user can interact with multiple bodies in multiple spaces. Recent teleoperation technologies have allowed a user controlling multiple objects simultaneously, but at the same time, it also exhibited a significant challenge, which can be attributed to the high cognitive load caused by switching and recogning various spaces/perspectives repeatedly and instantly. To tackle this challenge, by taking advantage of pre-attentive visual cues for users’ simultaneous information processing, we designed and evaluated a new user interface, called Shadow Clones, that projects self-body information in unattended areas for increasing the awareness of body-space relationships and allowing users to seamlessly switch across different visual perspectives from avatars or remote robots. We then explored the proposed approach through a simple visual reaching task with a performance evaluation in terms of task completion time and success rate. The results showed superior performance when compared with a condition that presents no projections of users’ body movements in unattended areas. We conclude by discussing possible mechanisms of this enhancement as well as two potential scenarios using the shadow clones approach, including new entertainment content for virtual reality e-sports and multiple robot teleoperation such as in a construction site or a disaster site, without compromising operational performance.
- Reem Alzahabi and Matthew S Cain. 2021. Ensemble perception during multiple-object tracking. Atten. Percept. Psychophys. 83, 3 (April 2021), 1263–1274.Google Scholar
Cross Ref
- A T Bahill, A Brockenbrough, and B T Troost. 1981. Variability and development of a normative data base for saccadic eye movements. Invest. Ophthalmol. Vis. Sci. 21, 1 Pt 1 (July 1981), 116–125.Google Scholar
- Renaud Blanch and Michaël Ortega. 2009. Rake cursor: improving pointing performance with concurrent input channels. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1415–1418.Google ScholarDigital Library
- Yuanzhi Cao, Tianyi Wang, Xun Qian, Pawan S Rao, Manav Wadhawan, Ke Huo, and Karthik Ramani. 2019. GhostAR: A Time-space Editor for Embodied Authoring of Human-Robot Collaborative Task with Augmented Reality. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 521–534.Google ScholarDigital Library
- Li-Hung Chang, Kazuhisa Shibata, George J Andersen, Yuka Sasaki, and Takeo Watanabe. 2014. Age-related declines of stability in visual perceptual learning. Curr. Biol. 24, 24 (Dec. 2014), 2926–2929.Google ScholarCross Ref
- Linfeng Chen, Kazuki Takashima, Kazuyuki Fujita, and Yoshifumi Kitamura. 2021. PinpointFly: An Egocentric Position-control Drone Interface using Mobile AR. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21, Article 150). Association for Computing Machinery, New York, NY, USA, 1–13.Google ScholarDigital Library
- H Collewijn, C J Erkelens, and R M Steinman. 1988. Binocular co-ordination of human horizontal saccadic eye movements. J. Physiol. 404 (Oct. 1988), 157–182.Google Scholar
- Barbara Anne Dosher, Songmei Han, and Zhong-Lin Lu. 2010. Information-limited parallel processing in difficult heterogeneous covert visual search. J. Exp. Psychol. Hum. Percept. Perform. 36, 5 (Oct. 2010), 1128–1144.Google ScholarCross Ref
- John Duncan. 2010. The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour. Trends Cogn. Sci. 14, 4 (April 2010), 172–179.Google ScholarCross Ref
- Paul E Dux, Jason Ivanoff, Christopher L Asplund, and René Marois. 2006. Isolation of a central bottleneck of information processing with time-resolved FMRI. Neuron 52, 6 (Dec. 2006), 1109–1120.Google ScholarCross Ref
- Paul E Dux, Michael N Tombu, Stephenie Harrison, Baxter P Rogers, Frank Tong, and René Marois. 2009. Training improves multitasking performance by increasing the speed of information processing in human prefrontal cortex. Neuron 63, 1 (July 2009), 127–138.Google ScholarCross Ref
- Dylan F Glas, Takayuki Kanda, Hiroshi Ishiguro, and Norihiro Hagita. 2012. Teleoperation of Multiple Social Robots. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 42, 3 (May 2012), 530–544.Google ScholarDigital Library
- Arvid Guterstam, Dennis E O Larsson, Joanna Szczotka, and H Henrik Ehrsson. 2020. Duplication of the bodily self: a perceptual illusion of dual full-body ownership and dual self-location. R Soc Open Sci 7, 12 (Dec. 2020), 201911.Google ScholarCross Ref
- Suk Won Han and René Marois. 2013. The source of dual-task limitations: serial or parallel processing of multiple response selections?Atten. Percept. Psychophys. 75, 7 (Oct. 2013), 1395–1405.Google Scholar
- James B Heald, Máté Lengyel, and Daniel M Wolpert. 2021. Contextual inference underlies the learning of sensorimotor repertoires. Nature 600, 7889 (Dec. 2021), 489–493.Google ScholarCross Ref
- Christopher G Healey, Kellogg S Booth, and James T Enns. 1996. High-speed visual estimation using preattentive processing. ACM Trans. Comput. Hum. Interact. 3, 2 (June 1996), 107–135.Google ScholarDigital Library
- Christopher G Healey and James T Enns. 2012. Attention and visual memory in visualization and computer graphics. IEEE Trans. Vis. Comput. Graph. 18, 7 (July 2012), 1170–1188.Google ScholarDigital Library
- Yoko Higuchi, Yoshiyuki Ueda, Kazuhisa Shibata, and Jun Saiki. 2019. Spatial variability induces generalization in contextual cueing. J. Exp. Psychol. Learn. Mem. Cogn. (Nov. 2019).Google Scholar
- Matthias Hoppe, Andrea Baumann, Patrick Chofor Tamunjoh, Tonja-Katrin Machulla, Paweł W Woźniak, Albrecht Schmidt, and Robin Welsch. 2022. There Is No First- or Third-Person View in Virtual Reality: Understanding the Perspective Continuum. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22, Article 360). Association for Computing Machinery, New York, NY, USA, 1–13.Google ScholarDigital Library
- Hiroshi Imamizu and Mitsuo Kawato. 2009. Brain mechanisms for predictive control by switching internal models: implications for higher-order cognitive functions. Psychol. Res. 73, 4 (July 2009), 527–544.Google ScholarCross Ref
- Shunichi Kasahara, Ryuma Niiyama, Valentin Heun, and Hiroshi Ishii. 2013. exTouch: spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (Barcelona, Spain) (TEI ’13). Association for Computing Machinery, New York, NY, USA, 223–228.Google ScholarDigital Library
- A M Clare Kelly and Hugh Garavan. 2005. Human functional neuroimaging of brain changes associated with practice. Cereb. Cortex 15, 8 (Aug. 2005), 1089–1102.Google ScholarCross Ref
- Masatomo Kobayashi and Takeo Igarashi. 2008. Ninja cursors: using multiple cursors to assist target acquisition on large screens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 949–958.Google ScholarDigital Library
- Andrey Krekhov and Jens Krueger. 2021. Deadeye: A Novel Preattentive Visualization Technique Based on Dichoptic Presentation. (Jan. 2021). arxiv:2101.07048 [cs.HC]Google Scholar
- Manu Kumar, Andreas Paepcke, and Terry Winograd. 2007. EyeExpos’e: Switching applications with your eyes.Google Scholar
- Seungyeol Lee and Jeon-Il Moon. 2017. Management of a Single-User Multi-Robot Teleoperated System for Maintenance in Offshore Plants. In Multi-agent Systems. InTech.Google Scholar
- Bingxin Li, Xiangqian Li, Gijsbert Stoet, and Martin Lages. 2019. Exploring individual differences in task switching. Acta Psychol. 193 (Feb. 2019), 80–95.Google Scholar
- René Marois and Jason Ivanoff. 2005. Capacity limits of information processing in the brain. Trends Cogn. Sci. 9, 6 (June 2005), 296–305.Google ScholarCross Ref
- Antonella Maselli and Mel Slater. 2014. Sliding perspectives: dissociating ownership from self-location during full body illusions in virtual reality. Front. Hum. Neurosci. 8 (Sept. 2014), 693.Google Scholar
- B McElree and M Carrasco. 1999. The temporal dynamics of visual search: evidence for parallel processing in feature and conjunction searches. J. Exp. Psychol. Hum. Percept. Perform. 25, 6 (Dec. 1999), 1517–1539.Google ScholarCross Ref
- Reiji Miura, Shunichi Kasahara, Michiteru Kitazaki, Adrien Verhulst, Masahiko Inami, and Maki Sugimoto. 2022. MultiSoma: Motor and Gaze Analysis on Distributed Embodiment With Synchronized Behavior and Perception. Frontiers in Computer Science 4 (2022).Google Scholar
- Yukiya Nakanishi, Masaaki Fukuoka, Shunichi Kasahara, and Maki Sugimoto. 2022. Synchronous and Asynchronous Manipulation Switching of Multiple Robotic Embodiment Using EMG and Eye Gaze. In Augmented Humans 2022 (Kashiwa, Chiba, Japan) (AHs 2022). Association for Computing Machinery, New York, NY, USA, 94–103.Google ScholarDigital Library
- Ulric Neisser and Robert Becklen. 1975. Selective looking: Attending to visually specified events. Cogn. Psychol. 7, 4 (Oct. 1975), 480–494.Google ScholarCross Ref
- Mikhail Ostanin, Stanislav Mikhel, Alexey Evlampiev, Valeria Skvortsova, and Alexandr Klimchik. 2020. Human-robot interaction for robotic manipulator programming in Mixed Reality. In 2020 IEEE International Conference on Robotics and Automation (ICRA). 2805–2811.Google ScholarCross Ref
- H Pashler. 1994. Dual-task interference in simple tasks: data and theory. Psychol. Bull. 116, 2 (Sept. 1994), 220–244.Google ScholarCross Ref
- H Pashler. 1994. Graded capacity-sharing in dual-task interference?J. Exp. Psychol. Hum. Percept. Perform. 20, 2 (April 1994), 330–342.Google Scholar
- Z W Pylyshyn and R W Storm. 1988. Tracking multiple independent targets: evidence for a parallel tracking mechanism. Spat. Vis. 3, 3 (1988), 179–197.Google ScholarCross Ref
- Kari-Jouko Räihä and Oleg Špakov. 2009. Disambiguating ninja cursors with eye gaze. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1411–1414.Google ScholarDigital Library
- Jonas Schjerlund, Kasper Hornbæk, and Joanna Bergström. 2021. Ninja Hands: Using Many Hands to Improve Target Selection in VR. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–14.Google ScholarDigital Library
- E H Schumacher, T L Seymour, J M Glass, D E Fencsik, E J Lauber, D E Kieras, and D E Meyer. 2001. Virtually perfect time sharing in dual-task performance: uncorking the central cognitive bottleneck. Psychol. Sci. 12, 2 (March 2001), 101–108.Google ScholarCross Ref
- Mariano Sigman and Stanislas Dehaene. 2008. Brain mechanisms of serial and parallel processing during dual-task performance. J. Neurosci. 28, 30 (July 2008), 7585–7598.Google ScholarCross Ref
- Kazuma Takada, Midori Kawaguchi, Yukiya Nakanishi, Akira Uehara, Mark Armstrong, Adrien Verhulst, Kouta Minamizawa, and Shunichi Kasahara. 2021. Parallel Ping-Pong: Demonstrating Parallel Interaction through Multiple Bodies by a Single User. In SIGGRAPH Asia 2021 Emerging Technologies(Tokyo, Japan) (SA ’21 Emerging Technologies, Article 12). Association for Computing Machinery, New York, NY, USA, 1–2.Google Scholar
- Kazuma Takada, Midori Kawaguchi, Akira Uehara, Yukiya Nakanishi, Mark Armstrong, Adrien Verhulst, Kouta Minamizawa, and Shunichi Kasahara. 2022. Parallel Ping-Pong: Exploring Parallel Embodiment through Multiple Bodies by a Single User. In Augmented Humans 2022 (Kashiwa, Chiba, Japan) (AHs 2022). Association for Computing Machinery, New York, NY, USA, 121–130.Google ScholarDigital Library
- James T Townsend. 1990. Serial vs. Parallel Processing: Sometimes They Look like Tweedledum and Tweedledee but they can (and Should) be Distinguished. Psychol. Sci. 1, 1 (Jan. 1990), 46–54.Google Scholar
- A M Treisman and G Gelade. 1980. A feature-integration theory of attention. Cogn. Psychol. 12, 1 (Jan. 1980), 97–136.Google ScholarCross Ref
- Adrien Verhulst, Yasuko Namikawa, and Shunlchl Kasahara. 2022. Parallel Adaptation: Switching between Two Virtual Bodies with Different Perspectives Enables Dual Motor Adaptation. In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 169–177.Google ScholarCross Ref
- C Ware. 2004. Information Visualization: Perception for Design: Second Edition. unknown.Google ScholarDigital Library
- Kei Watanabe and Shintaro Funahashi. 2014. Neural mechanisms of dual-task interference and cognitive capacity limitation in the prefrontal cortex. Nat. Neurosci. 17, 4 (April 2014), 601–611.Google ScholarCross Ref
- Jeremy M Wolfe. 2014. Approaches to Visual Search. In The Oxford Handbook of Attention, Anna C. (Kia) Nobre and Sabine Kastner (Eds.). Oxford University Press.Google Scholar
Index Terms
- ShadowClones: an Interface to Maintain a Multiple Sense of Body-space Coordination in Multiple Visual Perspectives
Recommendations
Shadow Clones: an Unattended Visual Interface to Enhance the Task Performance in Multiple Space using Gaze-Switching
SUI '22: Proceedings of the 2022 ACM Symposium on Spatial User InteractionWe propose Shadow Clones; a user interface with unattended visual information to accomplish multiple tasks in multiple spaces using fast-gaze-switching. Shadow Clones interface provides not only a cursor on the view area the user is gazing at but also ...
Salient object detection via multiple saliency weights
Salient object detection aims to emulate the extraordinary capability of human visual system, which has the ability to find the most visually attractive objects in a complex visual scene. The human visual attention is often complicated and affected by ...
Overview Use in Multiple Visual Information Resolution Interfaces
In interfaces that provide multiple visual information resolutions (VIR), low-VIR overviews typically sacrifice visual details for display capacity, with the assumption that users can select regions of interest to examine at higher VIRs. Designers can ...
Comments